Learning ONNX for trading - page 12

[MetaQuotes]

Digit classification on CPU with ONNX Runtime demo

Digit classification on CPU with ONNX Runtime demo

Open Neural Network Exchange (ONNX) provides an open-source format for both deep learning and machine learning models. We can train our models in whichever framework we prefer and then convert the model to ONNX format. With Microsoft's ONNX Runtime, we can run an inference session with onnx models on any environments which give us slightly faster implementation. Here is a simple demonstration of the same. The model is trained to recognize digits using the MNIST dataset with PyTorch. I'm running an inference session on Linux CPU.

https://github.com/NagarajSMurthy/Digit-recognizer

[MetaQuotes]

Billions of NLP Inferences on the JVM using ONNX and DJL

Billions of NLP Inferences on the JVM using ONNX and DJL

The CTO of a media intelligence company discusses how they use the JVM and DJL and Hugging Face for NLP tokenization in the machine learning pipeline to mine the media landscape for various use cases. As their product features have driven towards it, their machine learning and modeling system have become essential parts to keep everything running as they have reached a scale where CPU couldn't suffice anymore. They've switched from using a 32-bit floating-point model to 16-bit, which led to a 3% increase in effectiveness but faced conversion errors and rare memory leaks during the process, which they resolved by replacing several implementations. They invested in robustness by adding GPU-powered CI, and setting up an advanced Prometheus logic stack that monitors various inferences' latency and tokenization latency. Their future plans include improving GPU efficiency and adding more models to the system by creating a multi-GPU setup.

• 00:00:00 In this section of the video, Viet, the CTO of a media intelligence company, discusses how his company's infrastructure is based on the JVM and how they use it to mine the media landscape for various use cases such as product launch tracking and reputation tracking. He explains that their system is powered by JVM, and they use DJL and Hugging Face for NLP tokenization in the machine learning pipeline. The company's machine learning and modeling system are no longer just nice-to-haves but essential parts to keep everything up and running as their product features drive towards it. They've reached a scale where CPU couldn't suffice anymore, so they look at Cubernates as a system to horizontal scale it. Their system is yielding billions of inferences per day and they do GPU acceleration and use 8 million sites to build a whole web crawler system.
  
  
• 00:05:00 In this section, the speaker discusses their experience while working with the ONNX ecosystem. They talk about the switch from using 32-bit floating-point model to 16-bit, which led to a 3% increase in effectiveness. However, they faced conversion errors during this process, which resulted in unexpected output from the ONNX model. They also encountered rare memory leaks that they resolved by replacing several implementations. The speaker explains that they invested in robustness by adding GPU-powered CI and setting up an advanced Prometheus logic stack that monitors various inferences' latency and tokenization latency. Finally, they discuss their future plans, which include improving GPU efficiency and adding more models to the system by creating a multi-GPU setup.

[MetaQuotes]

ONNX and the JVM

ONNX and the JVM

ONNX support on the Java Virtual Machine (JVM) is crucial as ML models are becoming more prominent in virtually every application. With Java being one of the largest platforms used in building live software applications, it's essential to provide support within programming languages, such as Java or C#. Oracle aims to provide the ONNX runtime C API in Java, enabling easy deployment with minimal performance impact through the use of a thin layer of the Java API over the C API. The speaker also discusses an open-source library for writing ONNX models from Java, introduces a logistic regression graph example, and invites contribution to the ONNX export stuff in Trippo while discussing the lack of standardization in ONNX metadata format.

• 00:00:00 In this section, Adam Pocock, a researcher in the machine learning research group at Oracle Labs, explains why supporting ONNX on the Java Virtual Machine (JVM) is important. With ML models becoming more important in almost every application, it is critical to provide support within the languages that applications are written in, such as Java or C#. According to Pocock, Java is one of the largest platforms for building live software applications worldwide, with millions of Java developers. As a result, Oracle's aim is to provide the ONNX runtime C API in Java, making it easy to deploy for users, with minimal performance impact. A quick code example is shown to demonstrate the thin layer of the Java API over the C API.
  
  
• 00:05:00 In this section, the speaker discusses the challenges of using Java objects with a paired native object in order to minimize overheads for machine learning models. They explain that there are safety nets in place to ensure that everything is cleaned up, but using regular arrays is not recommended. The speaker also introduces an open-source library for writing ONNX models from Java, which includes attribute validation and checking and the ability to easily export Java arrays. They provide an example of how to write a logistic regression graph using this library and mention plans to expand the operator enum and integrate provenance and metadata into the converted models.
  
  
• 00:10:00 In this section, the speaker discusses the lack of standardization concerning the metadata format in ONNX. Although there is the possibility of including data such as training data and hyperparameters in ONNX metadata, it is not standardized. The speaker expresses interest in the work that Intel is doing to standardize the metadata format. The speaker also invites any contributions to the ONNX export stuff in Trippo, and encourages discussion on the topic.

[MetaQuotes]

Build your high-performance model inference solution with DJL and ONNX Runtime

Build your high-performance model inference solution with DJL and ONNX Runtime

The Deep Java Library (DJL) is a machine learning library built on Java that abstracts deep learning libraries and offers multiple backends, such as Apache MXNet, TensorFlow, and PyTorch. The library has a set of pre-trained models for various tasks and is service-ready, having undergone rigorous testing to ensure top performance while having control over memory usage. The speakers also introduce the hybrid engine concept that loads both engines together, offering a smoother transition between engines for inference. Further developments include supporting ARM servers, running ONNX Runtime on Android devices, and bringing the hybrid engine solution to edge devices.

• 00:00:00 In this section, the speaker introduces the Deep Java Library (DJL) and its integration with the ONNX Runtime, which has proven to have good performance for machine learning use cases both within and outside of Amazon. DJL is a machine learning library built on Java that abstracts deep learning libraries and offers multiple backends, such as Apache MXNet, TensorFlow, and PyTorch. DJL has a set of pre-trained models, including popular ones from Hugging Face and Torch Hub, for tasks such as image classification, object detection, sentiment analysis, and action recognition. The advantage of DJL is that it is service-ready and has undergone rigorous testing to ensure top performance while having control over memory usage. The speaker also shares some successful customer use cases that leveraged ONNX Runtime and DJL, showcasing significant improvements in model inference speed and multi-threading inference, even when converting from different deep learning frameworks such as TensorFlow and PaddlePaddle, which did not support thread-safe multi-threading.
  
  
• 00:05:00 In this section, the speakers discuss the challenges faced by users of ONNX Runtime due to the lack of built-in operators, making it difficult to perform pre-processing and post-processing in Java for the un-dimensional array operation. They propose a new concept called the hybrid engine, which loads both engines together simultaneously, allowing for smoother transition between engines for inference. The Deep Java Library offers a criteria class where the user can define how their model will be loaded, and the translator provides abstraction to perform pre-processing and post-processing. They also introduce the ND Manager, a tree-like architecture that replaces the garbage collection system in Java to provide cost-effective memory collection. The speakers mention further developments, such as supporting the ARM server, running ONNX Runtime on Android devices, and bringing the hybrid engine solution to edge devices.

[MetaQuotes]

[FlexFlow Bootcamp 2020] FlexFlow Front-End Supports: TensorFlow Keras, PyTorch, ONNX, and more

[FlexFlow Bootcamp 2020] FlexFlow Front-End Supports: TensorFlow Keras, PyTorch, ONNX, and more

In this section of the video, the speaker discusses the FlexFlow Python API, which supports TensorFlow Keras, PyTorch, and ONNX. The process of creating and training a model involves adding operators to the model, compiling the model, creating data loaders, and initializing/training the model using the fit function or customized training procedures. The speakers also discuss support for Keras and PyTorch models in FlexFlow, as well as the ability to import pre-existing models through the ONNX intermediate representation. However, it is important to ensure consistency between the library used to build FlexFlow and the one used to build the ONNX python package.

• 00:00:00 In this section of the video, the presenters introduce the front-end support in FlexFlow, specifically the Python API. The FlexFlow runtime is built on top of the Legion runtime, which is a task-based program system developed by researchers from Stanford University, NVIDIA, and Los Alamos. The Python API calls the C++ API through the cffi library, and a thin layer of Python API provides the same performance as using the C++ API. The presenters then discuss the steps to create a model with the FlexFlow Python API, which involves first creating a configuration file and then creating an input tensor in NCHW format. Additionally, they introduce support for Keras and ONNX.
  
  
• 00:05:00 In this section of the video, the speaker discusses the process of creating and training a model using the FlexFlow Python API, which supports TensorFlow Keras, PyTorch, and ONNX. The speaker begins by explaining how to add operators to the model, such as the convolution 2D operator, which always takes an input tensor and returns an output tensor. The next step is to compile the model, which allocates tensors and optimizes the parallelization strategy and model fusion. Then, data loaders are created, and the model is initialized and trained using the fit function or customized training procedures. The speaker also highlights the similarities between the Python and C++ APIs and presents a list of operators supported by FlexFlow, with the ability to implement new operators upon request. Finally, the speaker discusses the support for Keras and PyTorch models in FlexFlow, with a demo scheduled for the hands-on exercise.
  
  
• 00:10:00 In this section, the speakers discuss importing pre-existing models into FlexFlow, specifically through the ONNX intermediate representation. The ONNX package allows for models to be exported from other deep learning frameworks, such as PyTorch or TensorFlow, and imported into FlexFlow using the same file name. However, it is important to ensure that the library used to build FlexFlow is the same as the one used to build the ONNX python package. ONNX support is still a work in progress, and patches are welcome.

[MetaQuotes]

Learning Machine Learning with .NET, PyTorch and the ONNX Runtime

Learning Machine Learning with .NET, PyTorch and the ONNX Runtime

In this video about learning machine learning with .NET, PyTorch, and the ONNX Runtime, the speakers introduce the ONNX Runtime and explain the different steps for training a machine learning model. They also demonstrate how to use the ONNX format with .NET for deep learning and discuss the importance of understanding hyperparameters and the optimization method for accurate model predictions. The speakers also show how to use the ONNX runtime to load a model and make predictions, as well as how to handle any potential errors with a try-block when running a session. Additionally, they discuss the use of the unsuredness vector to show the AI's uncertainty in its predictions and mention some industries where AI is being used, such as fraud detection and recommendation systems.

• 00:00:00 In this section of the video "Learning Machine Learning with .NET, PyTorch and the ONNX Runtime", the hosts introduce the ONNX Runtime and how it can be used with .NET for deep learning. They explain that once a model is trained, the next step is to bring the model file into the project and get the answer. They use the example of recognizing numbers and show a Py Torch GitHub repo with a Convolutional Neural Network model. They also highlight the usefulness of VS Code as an IDE for Python, with its built-in features and functionality.
  
  
• 00:05:00 In this section, the speaker discusses how they use test data to train their machine learning model, and once it has figured out the best way to adjust the knobs, they save the model and turn off the water. They demonstrate an example of a model they trained with images of numbers zero through nine, explaining that there are ten elements in the output because the images are only zero through nine. They also emphasize the importance of the loss function, which measures the difference between the predicted outcome and actual outcome, and the accuracy, which measures how many outcomes are correct or incorrect. The speaker suggests that picking the right model shape, hyperparameters like learning rate, batch size, and epochs, and testing on different data sets are essential steps to ensure the machine learning algorithm achieves high accuracy.
  
  
• 00:10:00 In this section of the video, the speakers discuss the use of the ONNX format, which is described as the "PDF for neural networks." They demonstrate the creation of an ONNX file and then show how to use it within a .NET application using the ONNX runtime. The use of the inference session global variable is explained, in order to avoid the need to reload the file for each inference. The importance of understanding the model's optimization method and hyper-parameters is also highlighted. Finally, the size differences between models are discussed and how this affects the difficulty of making predictions.
  
  
• 00:15:00 In this section, the speaker explains how to use the ONNX runtime to load a model and make predictions, specifically in the context of computer vision. They demonstrate using an example of drawing the number 3 and converting it to a 28 by 28 pixel image, followed by creating a single float array and taking the average of all the colors while subtracting 255. The speaker also discusses the use of tensor of t, a data type used by the ONNX runtime, for efficient indexing into memory and the importance of normalizing the image for proper input. They also show how to create a tensor from the named ONNX value and how to handle any potential errors with a try-block when running a session.
  
  
• 00:20:00 In this section, the speakers are demonstrating the process of using PyTorch and .NET in recognizing numbers using machine learning. The system identifies digits as matrices and is trained to identify specific digits using optimization algorithms. The system doesn't have the capability to say "I don't know" and instead is always going to give a best effort answer, even though it may be wrong when presented with unfamiliar input. The final output is a vector of probabilities, which is then transformed into a human-readable format.
  
  
• 00:25:00 In this section, the speaker discusses how an unsuredness vector can be used to show the AI's uncertainty in its predictions. The speaker notes that this feature is only available if it was trained in such an unsuredness dataset. The speaker then clarifies the difference between training and inference stages in machine learning and how they are separated in ML.NET. The speaker also mentions that ONNX format can be used in .NET applications and that it is usually used for more complex datasets like neural network models. Finally, the speaker mentions that AI is currently being used in various industries like fraud detection and recommendation systems.

[MetaQuotes]

How to Read and Write an ONNX Model in ML.NET

How to Read and Write an ONNX Model in ML.NET

The video starts by introducing ONNX - an open format created by Microsoft and Facebook that allows the exchange of machine learning models between different frameworks. The presenter explains how ML.NET, an open-source and cross-platform machine learning library, has support for ONNX models. The video then proceeds to show how to create and export an ML.NET model into an ONNX file, using the ONNX Runtime package. Once the ONNX model is created, the video explains how to use it to make predictions on new data in ML.NET. Overall, the video provides a comprehensive guide on how to use ONNX models with ML.NET for machine learning applications.

[MetaQuotes]

Integrating scikit-learn ML Models with ML.NET Using ONNX - Practical ML.NET User Group 02/18/2022

Integrating scikit-learn ML Models with ML.NET Using ONNX - Practical ML.NET User Group 02/18/2022

In this video, the speaker discusses the integration of Scikit-learn machine learning models with the .NET ecosystem using ONNX. They use lead scoring in the digital marketing field as a practical example of how to build, deploy and test machine learning models for client systems. The presenter explains the lead scoring process and emphasizes the importance of building an automated tool that maximizes the efficiency of marketing and sales teams. The speaker discusses the challenge of deploying machine learning models for client systems and introduces ONNX as a solution. They provide an overview of the tools, packages and techniques used for integrating Scikit-learn ML models with ML.NET using ONNX. The speaker demonstrates how to build and serialize a logistic regression model, convert it to ONNX format, and run the ONNX model, before integrating it with the .NET ecosystem using Azure Functions. Overall, this video serves as a practical guide for developers looking to integrate Scikit-learn ML models with the .NET ecosystem using ONNX.

In this Practical ML.NET User Group session, the presenter demonstrates the use of the ONNX format to create a lead scoring ONNX model that can be incorporated into the Dot Net ecosystem. The implementation can be used in parallel with ML.NET, enabling the execution of ONNX models using the ONNX runtime while performing machine learning using ML.NET. The presenter shares a GitHub repository that contains the techniques used, libraries, and step-by-step instructions for building the ONNX model. The use of the ONNX format allows for a cross-platform runtime engine and helps to bridge the gap between data scientists and application developers. The session's value lies in the practical implementation of a proof of concept system, which can be used with other algorithms.

• 00:00:00 In this section, the speaker introduces their topic of integrating Scikit-learn machine learning models into the .NET ecosystem and demonstrates a practical example of how to do this. They also discuss the Open Neural Network Exchange Format (ONNX) and how it helps deploy models within the .NET ecosystem. The speaker introduces their team's project of lead scoring in the digital marketing field, using their business domain knowledge to develop a complementary tool that automates the lead scoring process. They provide a brief overview of the lead scoring process and emphasize the importance of building a system that is up and running at the end of the day.
  
  
• 00:05:00 In this section, the speaker explains the process of lead conversion and the need for machine learning in optimizing the lead scoring and generation process. The speaker highlights the importance of big data generated by digital marketing and sales teams, and the need for a predetermined structured format for the data. By using machine learning, they can adopt a supervised approach to retrieve historical data and improve the lead conversion process. The automated tool aims to maximize the efficiency of the marketing and sales teams by providing insights from the data, such as identifying the most important features and fine-tuning the marketing strategy to generate more leads. The prototype involves retrieving data from different sources and implementing traditional data science techniques such as data cleaning, exploratory data analysis, and data preparation before applying the machine learning model.
  
  
• 00:10:00 In this section of the video, the speaker discusses the process of generating data insights and building machine learning models, as well as the challenge of deploying these models to be used by clients and companies. They introduce ONNX (Open Neural Network Exchange) as a possible solution to this deployment challenge. ONNX is a file format that represents models in the form of graphs and is framework-agnostic, meaning it can be shared between different machine learning tools and frameworks. The ONNX runtime is a set of APIs for integrating models into target environments and is cross-platform and high-performance. The speaker provides links to the ONNX website and GitHub for more information.
  
  
• 00:15:00 In this section, the speaker discusses the challenge of integrating machine learning models into client systems after they have been developed using data science tools like Python. They recommend using ONNX as a unified format for the models and demonstrate how to import, read, build, deploy, and test the model in a .NET ecosystem. The speaker also mentions a publicly available lead scoring dataset that can be used to build a prototype for data-driven marketing. They discuss techniques such as data cleaning, exploratory data analysis, and supervised machine learning, including regression and classification for lead scoring scenarios. The goal is to maximize efficiency for marketing and sales teams and help boost performance by focusing on the most potential clients.
  
  
• 00:20:00 In this section of the video, the speaker provides an overview of the tools and packages they are using for integrating scikit-learn ML Models with ML.NET using ONNX. They mention that they are using Anaconda distribution for Python computing, Jupyter notebook for building the models with Python-based scikit-learn library, and .NET version 6 with ONNX version 1.9 and ONNX runtime version 181. The speaker briefly goes over the data set they are using and the techniques they implemented for data wrangling, exploratory data analysis, and performance evaluation metrics. They also mention that they built a logistic model with over 82% accuracy using the logistic regression for classification purposes using the Lipliner algorithm.
  
  
• 00:25:00 In this section, the presenter discusses the process of building a logistic regression model for predicting lead scoring, and then serializing it using the pickle library in Python to save it as a pkl file. Afterward, the presenter loads the pkl file to test the model, and the result shows an accuracy of 0.76 on the test set. The presenter then demonstrates how to convert the model from scikit-learn to the ONNX format, which is necessary for deploying the model in the Microsoft.NET ecosystem. This section can serve as a practical guide on how to integrate scikit-learn machine learning models with ML.NET using ONNX.
  
  
• 00:30:00 In this section, the speaker discusses the process of integrating scikit-learn ML models with ML.NET using ONNX. They suggest checking the list of supported algorithms by ML.NET's ONNX converters before implementing any machine learning, and demonstrate how to import the SK2ONNX supported converters. They then define the path to store the model and the number of features, emphasizing the importance of the schema in naming inputs, and converting them to flow tensor type. Finally, they show how to build the ONNX model using the convert_sklearn function, using the model and the initial types previously defined.
  
  
• 00:35:00 In this section, the speaker demonstrates how to save a Python-based scikit-learn machine learning model in ONNX format and how to run the ONNX model using the ONNX runtime's inference session. The input name of the ONNX model is identified, and the signature output names are not of interest for this demonstration. Additionally, by using the predict_proba method, the probabilities of being or not being converted are extracted, and they are used as indicators and specific scenario weights for the lead scoring model.
  
  
• 00:40:00 In this section, the speaker begins by creating a blank solution in Visual Studio 2022 for integrating a scikit-learn ML model, specifically a logistic regression model, with the .NET ecosystem using ONNX. The speaker then creates an Azure function in the solution and gets the boilerplate code for implementing it with .NET 6 as an HTTP trigger. They use Insomnia to create a new request with JSON format input features, which will be parsed and used for the system's integration. The speaker installs the Microsoft ML ONNX runtime package from NuGet and creates a class to explain the input feature set. The speaker then uses code snippets to speed up the integration process.
  
  
• 00:45:00 In this section, the speaker discusses how to retrieve and deserialize the request body from an API using the JSON converter, in order to send it as an input to an ML model. They create an object of the LSLead Scoring Model parameter class and assign values from the deserialized request body to it. They also explain the importance of implementing input tensor and converting input data into a TensorFlow object of the tensor, so it can be incorporated in the inference session. Finally, they create an inference session using the ONNX runtime and retrieve the session output to run the ML model.
  
  
• 00:50:00 In this section, the presenter attempts to show how to integrate scikit-learn ML models with ML.NET using ONNX, but encounters technical difficulties with Visual Studio. After attempting to fix the issue, the presenter resorts to showing the code snippets and discusses the necessary packages and input tensors needed for the integration. Despite the setbacks, the presenter is able to provide a link to the function and input feature set used in the demo for viewers to access and try on their own.
  
  
• 00:55:00 In this section, the presenter demonstrates how to use ONNX to integrate scikit-learn machine learning models with ML.NET. They first show how to unwrap the probabilities output from the predict-probability implementation and convert them into a dictionary of long and float values representing the probabilities of being or not being converted. They then use these probabilities to obtain a lead score for being converted and not being converted, which they display in an Insomnia message. Finally, they use a Jupyter notebook to show how a record from the test set is sent to the model and retrieve the corresponding output value of one, indicating that the individual is likely to be converted.
  
  
• 01:00:00 In this section, the presenter demonstrates how the use of the Open Neural Network Exchange (ONNX) format allows for the creation of a lead scoring ONNX model that can be implemented as part of another framework. The presenter shows that this can be done using TensorFlow, Keras, and Dot Net targeting framework. The presenter also shows that based on the resulting ONNX model, they were able to incorporate this file format into the Dot Net ecosystem and subsequently build a simple solution that could run the session and retrieve results. They also discuss how using the ONNX format allows for a cross-platform runtime engine and helps to bridge the gap between data scientists and application developers. The presenter shares a code repository on GitHub that includes a notebook document and workspace needed to start integrating ONNX models within the Dot Net ecosystem.
  
  
• 01:05:00 In this section, the presenter shares a GitHub repository that contains the techniques used, libraries, and step-by-step instructions for building the ONNX model, with the most important part being the incorporation of ONNX and ONNX runtime packages. The presenter also emphasizes that the implementation can be used in parallel with ML.NET, enabling the execution of ONNX models using the ONNX runtime while performing machine learning using ML.NET. Lastly, the presenter thanks the attendees for their patience and time and notes that the session's value lies in the practical implementation of a proof of concept system, which can be used with other algorithms.

[MetaQuotes]

Machine learning models with ONNX and .NET | .NET Conf 2022

Machine learning models with ONNX and .NET | .NET Conf 2022

The "Machine learning models with ONNX and .NET" video from .NET Conf 2022 introduces viewers to the concepts of AI and machine learning, including the difference between deep learning and traditional programming. The presenters provide an overview of Azure Machine Learning, PyTorch, and ONNX, and demonstrate how to create a pipeline using Azure Machine Learning to train machine learning models with ONNX and .NET. They also explain how to integrate a machine learning model into a .NET Maui application and discuss techniques to reduce the size of ONNX models for mobile devices. The section ends by introducing the next speaker, Rory, who will be discussing accessibility.

• 00:00:00 In this section of the video, Beatriz and Henk introduce AI and machine learning, and the difference between the two. They explain that machine learning is a subset of AI and has the ability to learn from data without being explicitly programmed with an algorithm. Instead, it produces a model based on the data provided, which can then be used to make predictions or decisions.
  
  
• 00:05:00 In this section, the speaker explains the difference between the training and prediction phases of machine learning. The prediction phase works like traditional programming where a model or algorithm is given input, produces an answer or prediction, and enables solving complex problems that are challenging to solve with traditional programming. The speaker also talks about deep learning, which is a subset of machine learning based on artificial neural networks inspired by the brain. The demo that the speaker presents in the video uses computer vision, a field of artificial intelligence that trains computers to interpret and understand the visual world, which includes classification, object detection, and image segmentation. Finally, the speaker discusses the concept of transfer learning, where a model trained for one task is used as a starting point for a different task, saving time in training the model.
  
  
• 00:10:00 In this section, the presenters give an overview of the two frameworks they will be using for their machine learning model: PyTorch and ONNX. PyTorch is an open-source deep learning framework that the presenters prefer for this demo, while ONNX is an open format built to represent machine learning models that enables AI developers to use models in a variety of frameworks. They explain that ONNX allows them to use PyTorch to create their model and then save it in ONNX, making it possible to read that model from within .NET MAUI. They also introduce Azure Machine Learning, which includes a set of cloud services that enable collaborative machine learning workloads, along with its key elements: experiments, datastores, environments, and model management. They explain that model management is the key to their story because they will be training their model in PyTorch and registering it in model management so others can check out and use it in the software of their choice. They also provide an overview of machine learning pipelines, which are workflows of steps that can use data sources, datasets, and run on compute targets.
  
  
• 00:15:00 In this section, the speaker discusses how to create a pipeline using the Azure Machine Learning Designer to train machine learning models with ONNX and .NET. The first step is to review the data, which is stored in Blob Storage. Next, they create a new pipeline using the designer, which enables them to drag and drop components onto the canvas. The components they select include the PyTorch model, the ONNX converter, and the model trainer. Once the components are defined, the speaker runs the pipeline on their Azure Machine Learning compute called BumbleBee. They demonstrate the features of the component definition and the code for the training file.
  
  
• 00:20:00 In this section, the presenter demonstrates how to use ONNX and .NET to create a machine learning model and integrate it into a .NET Maui application. The model was trained using Azure Machine Learning with Python and PyTorch and converted to ONNX format. The model was then registered in Model Management and used in a .NET application. The presenter explains the importance of allowing data scientists to work with their language of choice and developers to build applications in theirs. All sample codes used in the presentation can be found in the provided link, as well as other resources for learning more about machine learning and the AI developer role.
  
  
• 00:25:00 In this section of the video, the presenters discuss the file size of ONNX models and the techniques used to make them smaller and work better on mobile. They highlight that there is a huge area of machine learning dedicated to reducing the size of models, even if it means sacrificing some performance. They also mention a project called Seeing AI, which uses different AI models in a mobile app, and suggest that viewers check it out for themselves. Lastly, the presenters introduce Rory as the next speaker who will be discussing accessibility.

[MetaQuotes]

On .NET Live - Operationalizing ML models with ONNX, C# .... and Pokemon!

On .NET Live - Operationalizing ML models with ONNX, C# .... and Pokemon!

In this On.NET Live video, the hosts discuss operationalizing machine learning models with ONNX and bring in Cassie Kozyrkov as a special guest. Kozyrkov emphasizes the importance of mentorship and discusses using ONNX as a way to bridge the gap between data scientists and software engineers. The conversation covers various topics from creating a machine learning model using natural language processing and the importance of data transformation to testing with unseen data and deploying the model through Azure Functions. The speakers also discuss Azure Machine Learning and the resources available for those interested in exploring ONNX and machine learning models more broadly.

The On.NET Live video discusses operationalizing ML models with ONNX, C#, and (for fun) Pokemon. The first presenter talks about ONNX, a machine learning format that allows models to be saved and loaded across different frameworks, and how to operationalize the models using .NET. The second presenter discusses using ML.NET to create a Pokemon image classifier and show how it can be operationalized for deployment. Overall, the video provides a great overview of operationalizing machine learning models with ONNX and C#.

• 00:00:00 In this section of the video, the hosts introduce a special guest, Cassie, who will be discussing machine learning with Onnx and possibly even Pokémon. The conversation also includes announcements about upcoming conferences, such as .NET Conf and Start Dev Change, which was put on by Cassie and the team to help people learn new development skills and empower career switchers. The conference covered various technical fields with the aim of helping individuals find their passion and pursue a career change.
  
  
• 00:05:00 In this section, Cassie Kozyrkov talks about the importance of mentorship and the need for folks to engage in initiatives like Start Dev Change, especially during these times when people are struggling due to the pandemic. She then discusses ONNX, the open format for saving machine learning models and deploying them to different environments. Cassie explains that ONNX makes it easier to bridge the world between what data scientists are doing and what software engineers are doing, especially in cases where there is a disconnect between departments. With ONNX, the models can be passed off to the DevOps side, who can deploy it in a way that works best for the project, and it can be used in different applications, including C#, C++, and Python.
  
  
• 00:10:00 In this section, the speaker describes building a machine learning model using natural language processing to take data from wine descriptions and using it to predict the variety, points value, and price range of the wine. They mention using a huge library of data from wine magicians and building the model with different features. The model is created with Python and trained within Jupyter Notebooks. The speaker also explains how Pandas is used to manipulate the data and shows a word cloud to demonstrate the most common words in the data set.
  
  
• 00:15:00 In this section, the speaker discusses different approaches to machine learning and how to choose the appropriate one for a given task. They explain the difference between classical machine learning and deep learning, with deep learning requiring more layers and neural networks. The speaker recommends starting with scikit-learn, a classical machine learning library, and experimenting with different algorithms to see what works best. They also discuss the importance of data transformation and tokenization when processing text data, and how to balance model accuracy with model size.
  
  
• 00:20:00 In this section, the speaker discusses the data processing required for machine learning before actually training the model, such as data cleaning and feature extraction. They emphasize the importance of testing with unseen data and caution against relying solely on accuracy, as results can differ when applying the model to real-world production data. The speaker also demonstrates how to set up a data pipeline and use a logistic regression model to create a machine learning model, before moving on to the deployment stage.
  
  
• 00:25:00 In this section, the speakers discuss the natural language processing model used and the CountVectorizer algorithm used for it, which is part of the scikit-learn library. The pipeline shown in the video includes the CountVectorizer and logistic regression model. The authors explain how to export the model using the ONNX runtime and save it as an ONNX file. They go on to discuss the runtime's better performance, which takes into account the hardware it deploys to, giving it a significant speed advantage.
  
  
• 00:30:00 In this section, the presenters discuss the deployment of the model and the inference process. They mention that deploying a model involves considering the appropriate architecture and projects for the given situation, which can vary from using Azure machine learning to container resource or serverless offerings like Azure function. The model and code need to be uploaded to the cloud, and in this particular case, the model is stored in blob storage and then consumed by the function. The presenters also touch upon the importance of caching data to make the inference process faster and explain how the dictionary and inference session works to run the model in the function. While understanding the data type is necessary, one does not have to understand how the model was created to run it successfully.
  
  
• 00:35:00 not. In this section, the speaker discusses the process of operationalizing machine learning models with ONNX and how this can be done without the help of a data scientist. They emphasize the importance of understanding the input format and creating the tensor, and also explain the process of casting the model results into the desired format. The speaker mentions the need for simplification in the layers required to get to the model output, and how data can be cleaned and bucketed for better results. They also touch on the possibility of collecting metrics and feedback for retraining the model.
  
  
• 00:40:00 In this section, the speakers discuss the importance of testing and understanding how a machine learning model works in making predictions. They bring up the idea of creating unit tests and incorporating automated tests into the normal development process. Additionally, they mention the tools available in Azure Machine Learning for model interpretability, which can help in identifying potential issues with the model's predictions. Lastly, they touch upon deploying the model within an HTTP trigger through Visual Studio Code's extension.
  
  
• 00:45:00 In this section, the speakers discuss a fun wine-tasting game that was created using machine learning and deployed on Azure. They share the link to the website and talk about how the model was trained using sommelier descriptions from Winemag, and how they used Azure functions to deploy the model to Azure. They also discuss the wider uses of Azure machine learning, including tools for cloud compute, GPU acceleration, and model versioning, making it ideal for enterprise machine learning. Despite this, there may still be times when deploying to an endpoint is not necessary, and Azure machine learning allows for this simplicity.
  
  
• 00:50:00 In this section, the speaker discusses the various features and capabilities of Azure Machine Learning, which goes beyond just creating a model and consuming it through endpoints. They highlight the experimentation workbench, model versioning and compute, and DevOps plugins that allow for collaboration between data scientists and developers in the creation and deployment of machine learning models. They also mention the constantly evolving nature of the tool and recommend revisiting it if users haven't looked at it in a while. The speaker provides links to their GitHub repository and blog post for those interested in learning more about using ONNX with C# and deploying it through Azure Functions.
  
  
• 00:55:00 In this section, the speakers discuss various resources available for those interested in exploring ONNX and machine learning models in general. They recommend checking out the tutorial and example notebooks provided by ONNX, as well as exploring their open-source code on Github. Additionally, they briefly discuss using tools like Custom Vision to create computer vision models without the need for extensive code. Lastly, they joke about creating a Pokemon detection model in the future and recommend checking out .NET Conf happening in November for more machine learning announcements.