You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
How do you load data from an in-memory data set? #91
The only sample I can see uses the TextLoader to import data from a CSV file. How do you load in arbitrary data from e.g. a record or array and specify labels etc. on there?
Sorry, fat fingered issue.
As part of open API, we currently don't have it, but we working on it.
As part of internals we have ArrayDataViewBuilder and you can see example of it usage here:
@Ivanidzo4ka whichever way you go, make it as simple as possible to consume. This means either allowing F# Records / C# Classes, with either attributes or some guidance to specify labels / features etc., or (and this should definitely also be supported) support for naked arrays or sequences of either arrays (so a 2x2 array) or tuple data.
Looking for this as well. A couple use case I can see are:
Loaders other than flat files. SqlLoader, etc
2)Scenarios where the schema is not known at compile time - ie a command line tool that lets users pass in arbitrary csv file.
Hi @isaacabraham and @danroot , thanks for the feedback. @Ivanidzo4ka introduced an attempt to solve this issue in #106 , though I see that PR did not link to this issue but rather the pre-existing #10 . Is it possible for you to try it, and provide feedback on whether it does what you need?
Activity
Ivanidzo4ka commentedon May 9, 2018
Sorry, fat fingered issue.
As part of open API, we currently don't have it, but we working on it.
As part of internals we have ArrayDataViewBuilder and you can see example of it usage here:
machinelearning/test/Microsoft.ML.Predictor.Tests/TestTransposer.cs
Line 148 in c023727
isaacabraham commentedon May 9, 2018
Yep, understood. Not expecting everything to be there, but this is definitely something you want to have - the ability to pass in arbitrary data.
Ivanidzo4ka commentedon May 9, 2018
For one who will work on this in future.
we have DataViewConstructionUtils internal class which does desired.
machinelearning/src/Microsoft.ML.Api/DataViewConstructionUtils.cs
Line 17 in c023727
Maybe we need to add ArrayDataView, and we need to decide should we expose IHostEnvironment or not, and where in new API it belongs.
Ivanidzo4ka commentedon May 9, 2018
also copy of #10
isaacabraham commentedon May 10, 2018
@Ivanidzo4ka whichever way you go, make it as simple as possible to consume. This means either allowing F# Records / C# Classes, with either attributes or some guidance to specify labels / features etc., or (and this should definitely also be supported) support for naked arrays or sequences of either arrays (so a 2x2 array) or tuple data.
danroot commentedon May 17, 2018
Looking for this as well. A couple use case I can see are:
2)Scenarios where the schema is not known at compile time - ie a command line tool that lets users pass in arbitrary csv file.
TomFinley commentedon May 17, 2018
Hi @isaacabraham and @danroot , thanks for the feedback. @Ivanidzo4ka introduced an attempt to solve this issue in #106 , though I see that PR did not link to this issue but rather the pre-existing #10 . Is it possible for you to try it, and provide feedback on whether it does what you need?
isaacabraham commentedon May 17, 2018
Will give it a bash in the coming days - thanks!