Repository navigation
Error on prediction with LSTM Model in ONNX Format in ML.Net #5273
Description
System information
- Windows 10
- .NET Core 3.1, Microsoft.ML 1.5.0, Microsoft.ML.OnnxRuntime 1.3.0, Microsoft.ML.OnnxTransformer 1.5.0
Issue
I would like to predict Values with my LSTM Model in ONNX Format. I am struggling on data input for my ONNX Model in my ML.NET project. When I run my code, I get an Exception: System.NullReferenceException: 'Object reference not set to an instance of an object.' on prediction. The input data is just dummy. I am trying to find out, how to input data in form (None,3,7) as expected from LSTM Model. LSTM Model ist attached. Can someone help me, what I am doing wrong?
Why I get exception or how to deliver data for my ONNX Model.
using Microsoft.ML;
using Microsoft.ML.Data;
using System;
using System.Data.SqlClient;
namespace InQu.ML.Test.ONNX.FehlerAbschätzungLSTM
{
class Program
{
static void Main(string[] args)
{
var mlContext = new MLContext();
var x = new InputData[] {
new InputData() { X = new float[,] { { 1, 2, 3, 4, 5, 6, 7 }, { 1, 2, 3, 4, 5, 6, 7 }, { 1, 2, 3, 4, 5, 6, 7 } } }
};
IDataView dataView = mlContext.Data.LoadFromEnumerable<InputData>(x);
var pipeline = mlContext.Transforms.ApplyOnnxModel(modelFile: @".\scikit-learn\AbschatzungFehlerLSTM.onnx",
inputColumnNames: new[] { "input_layer", },
outputColumnNames: new[] { "dense_3" });
var model = pipeline.Fit(dataView);
var predEngine = mlContext.Model.CreatePredictionEngine<InputData, OutputData>(model);
OutputData prediction = new OutputData() { y = new float[0] };
predEngine.Predict(x[0], ref prediction);
}
}
public class InputData
{
[ColumnName("input_layer")]
[VectorType(3,7)]
public float[,] X { get; set; }
}
public class OutputData
{
[ColumnName("dense_3")]
[VectorType(1)]
public float[] y { get; set; }
}
}
Activity
[-]Consuming LSTM ONNX Model in ML.Net[/-][+]Error on prediction with LSTM Model in ONNX Format in ML.Net[/+]antoniovs1029 commentedon Jul 1, 2020
Hi, @ladodc . I believe this is a duplicate of dotnet/machinelearning-samples#828 So I'll close that issue, to continue the discussion here.
Can you please share the full stack trace of your exception? Also, if you're able to share your
.onnxmodel, it would be very useful so that we can take a closer look!At first look, I think the problem is that you're not correctly setting the input for your onnx model. Unfortunately, there's no general advice I could provide on how you should do this, since it will depend on the input expected by you
.onnxmodel.I'd think the problem is that you need to reshape your input
float[,]array into an array of the correct shape expected by onnx (you mention,(None, 3, 7)). The problem is that ML.NET doesn't currently have any transformer to do tensor reshaping, and it's necessary for users to actually implement their own reshape logic and use it inside aCustomMappingTransformer(see issue #765 ).In particular when working with Onnx, I gave this advice to a user that was having a similar problem like you (although in their case the problem was somewhat more complicated, as they were dealing with input images, and also needing to reshape that input). So you can check that advice to get an idea of how to use
CustomMappingTransformerfor your use case:#4795 (comment)
Note: I'm not sure on how reshape should be done if the onnx model expects a dimension of size
Nonesuch as in your case. I don't know if simply reshaping it to(1x3x7)would do, or maybe even(3x7). I'd need your.onnxmodel to actually experiment myself until I figure out what the correct logic should be for yourCustomMappingTransformerto work as expected.ladislav-dolezal commentedon Jul 2, 2020
Hi Antonio,
thanks for your response. I will try to follow your recommendation. Meanwhile I can give you my ONNX Model to experiment with this.
AbschatzungFehlerLSTM.zip
antoniovs1029 commentedon Jul 8, 2020
Hi, @ladodc .
So, after looking closer into this, it turns out that the
(None, 3, 7)input you have, will actually be pretty much treated as(1,3,7)by theOnnxTransformer. The first dimension is considered the batch dimension, so it can be ignored.Also, after playing around I realized that there's no need for the
CustomMappingTransformerthat I suggested in a previous comment, because you're actually already setting the input dimensions in the input class to match the ones expected by theOnnxTransformer. The real problem is that multidimensional input arrays aren't allowed in ML.NET, i.e. this is causing the exception:It should be:
The info of the shape is already set by the
VectorType(3,7)attribute, and ML.NET does require the users to input 1-D arrays. The exception you got is caused by this, and I'll try to open a PR soon so that a more understandable exception is thrown.After making the change above, I was able to run your code without problems, and got a predictions array. Still, the prediction was "0"... I tried modifying the values of the input vector, and I always got a prediction = "0". I did debug this, and it seems everything is working fine in the OnnxTransformer, so I'm not sure: Is this expected? Is your model supposed to return 0 with the data you provided?. I'm not yet sure what your model is supposed to do (the sigmoid at the end that returns 1 float, makes me think this is a binary classification problem), so please let us know if this expected behavior. If it's not expected, then please let us know what would be the expected behavior, and if possible any code you used to create the model, so that I can further investigate if this is a problem in ML.NET, in OnnxRuntime, or in your model. Thanks! 😄
antoniovs1029 commentedon Jul 10, 2020
I've opened a PR updating the docs of
VectorTypeAttribute(dims)so it's clearer that it's only meant to be used for 1-D arrays. And, as mentioned above, I was able to run your sample after flattening the input array. And so I'll close this issue now.Please, feel free to reopen the issue if you find that the behavior of running the Onnx model isn't as expected, and provide the info requested on my last comment if this is the case. Thanks. 😄