Repository navigation
Description
Issue
Passing a CollectionDataSource.Create<T>(data: var) to pipeline.Add(var) returns null values when referencing SQL Db via EF Core as the data source. Note that the latest version of ML.Net (0.20) is currently indicated as a dependency in the .csproj.
Inspecting the object indicates a valid state of data returned from the table but loaded as null into the pipeline however, I am not clear if the bug is attributable to the dataloader or its use. Any guidelines or help on the corrected implementation of parsing data from SQL Server is much appreciated.
Implementation
genericClass.cs
public partial class GenericClass {
public int CustomerId { get; set; }
public float PurchaseAmt { get; set; }
public string OrderType { get; set; }
}
public partial class GenericPredictionClass {
[ColumnName("Score")]
public float PredictedAmt;
}
sampleController.cs
public async Task<IActionResult> ShowOutput() {
PredictionModel<GenericClass, GenericPredictionClass> model = await TrainAsync();
return View();
}
private Task<List<GenericClass> LoadDataFromSql() {
var sqlTblOutput = _context.Table1
.Take(1000);
return Task.Run(function: () => sqlTblOutput.ToList());
}
private async Task<PredictionModel<GenericClass, GenericPredictionClass>> TrainAsync() {
var pipeline = new LearningPipeline();
List<GenericClass> parseData = await LoadDataFromSql();
var dataReference = CollectionDataSource.Create<GenericClass>(data: parseData);
pipeline.Add(dataReference);
pipeline.Add(new CategoricalOneHotVectorizer("OrderType"));
...
return model;
}
Locals
pipeline: {Microsoft.ML.LearningPipeline}
Rows [PipelineItemDebugRow[]]: {Microsoft.ML.PipelineItemDebugRow[10]}
[0] [PipelineItemDebugRow]: ""
Values [string]: ""
parseData [List]: Count = 1000
[0]: {Namespace.Models.GenericClass}
CustomerId: 1234
PurchaseAmt: 40.99
OrderType [string]: "Test Order"
dataReference [ILearningPipelineLoader]: {Microsoft.ML.Data.CollectionDataSource.ListDataSource<Namespace.Models.GenericClass>}
Non-Public members
_dataView [IDataView]: null
_dataViewEntryPoint [DataViewReference]: null
_listCollection [IList]: Count = 1000
[0]: {Namespace.Models.GenericClass}
CustomerId: 1234
PurchaseAmt: 40.99
OrderType [string]: "Test Order"
Log
"System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.ArgumentOutOfRangeException: Source column OrderType not found\nParameter name: Source\n at Microsoft.ML.Runtime.Data.OneToOneTransformBase.Bindings.Create(OneToOneTransformBase parent, OneToOneColumn[] column, ISchema input, ITransposeSchema transInput, Func2 testType)\n at Microsoft.ML.Runtime.Data.OneToOneTransformBase..ctor(IHostEnvironment env, String name, OneToOneColumn[] column, IDataView input, Func2 testType)\n at Microsoft.ML.Runtime.Data.TermTransform..ctor(ArgumentsBase args, ColumnBase[] column, IHostEnvironment env, IDataView input)\n at Microsoft.ML.Runtime.Data.CategoricalTransform.Create(IHostEnvironment env, Arguments args, IDataView input)\n at Microsoft.ML.Runtime.Data.Categorical.CatTransformDict(IHostEnvironment env, Arguments input)\n --- End of inner exception stack trace ---\n at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)\n at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)\n at Microsoft.ML.Runtime.EntryPoints.EntryPointNode.Run()\n at Microsoft.ML.Runtime.EntryPoints.EntryPointGraph.RunNode(EntryPointNode node)\n at Microsoft.ML.Runtime.EntryPoints.JsonUtils.GraphRunner.RunAllNonMacros()\n at Microsoft.ML.Runtime.EntryPoints.JsonUtils.GraphRunner.RunAll()\n at Microsoft.ML.LearningPipeline.Execute(IHostEnvironment environment)\n at Microsoft.ML.LearningPipelineDebugProxy.ExecutePipeline()"
System information
- Distro: Fedora 28 (latest)
- .NET Version: 2.1.301
- ML.NET Package version: 0.2.0
Activity
Ivanidzo4ka commentedon Jun 26, 2018
We currently don't support properties in "DTO classes", only fields. For the same reason we have problems with F# and other places which try to use classes with properties. And I'm not sure why.
Maybe @TomFinley know answer. Also @glebuk @shauheen for visibility.
TomFinley commentedon Jul 3, 2018
Sorry @Sarthakmalik and @Ivanidzo4ka , I somehow missed that I'd been tagged on a question till just now. Properties are a little more awkward due to inability to pass them via
ref, which is how buffer sharing works with pipelines (e.g., here). That's not an insurmountable problem of course: you couldgetthe property as a local variable, get the value, thensetthe property.There's no fundamental limitation other than limited engineering resources, I suppose.
Ivanidzo4ka commentedon Oct 18, 2018
We support via #616 .
So I'm closing this issue.