Repository navigation
how to use gpu? AddPredictionEnginePool #5985
Description
System Information (please complete the following information):
- OS & Version: ubuntu 20.04
- ML.NET Version: 1.6.0
- .NET Version: .net core 3.1
- Microsoft.Extensions.ML version: 1.6.0
- webapi
Describe the bug
cpu work well, but, how to use gpu?
Screenshots, Code, Sample Projects
Startup.cs:
public void ConfigureServices(IServiceCollection services)
{
services.AddControllers();
services.AddPredictionEnginePool<DeNoiseInput, DeNoiseOutput>().FromFile("./mlmodel/denoise_docs.zip");
}
Controller:
public class DocsController : Controller
{
private readonly PredictionEnginePool<DeNoiseInput, DeNoiseOutput> _denoiseEngine;
public DocsController(PredictionEnginePool<DeNoiseInput, DeNoiseOutput> denoiseEngine)
{
_denoiseEngine = denoiseEngine;
}
public ActionResult<string> DeNoise([FromBody] DeNoiseRequest request)
{
...
var outs = _denoiseEngine.Predict(imgExample);
...
}
}
Activity
luisquintanilla commentedon Nov 2, 2021
Hi @zcgg2zhmm
Currently using the GPU for inferencing is only supported for ONNX models. When using the ApplyONNX transform as part of your pipeline, you can set the
gpuDeviceIdparameter. Then, because the pipeline is saved as a .zip file, you don't need to do anything extra with thePredictionEnginePool. Just load it into your app and make predictions as you've shown in your code samples.xlg-go commentedon Nov 10, 2021
@luisquintanilla
Can you provide an example code? please!!
I want to use it in webapi.
Thanks a lot~