[camera] Access realtime image frames for pre-processing and effect overlays

[rubencg195]

Is it possible to access the real-time camera frame feed to do pre-processing or to be used for ML implementations like face-recognition, Image overlays, filters, etc.?

[zoechi]

Sounds like https://github.com/flutter/plugins/tree/master/packages/camera can do that but I haven't tried myself yet.

[rubencg195]

Thanks, but it will be better if you can obtain the byte array to work with instead of writing in memory and then retrieving the image for preprocessing. It can be costly in time and resources. I think it can be a really good feature to implement. Just think of the results, ML application, filters, AR/VR application, etc.

Maybe includying a Opencv wrapper for flutter. If possible, it could be awesome!
https://docs.opencv.org/2.4/doc/tutorials/introduction/android_binary_package/O4A_SDK.html

[dessant]

Have you found a workaround for this? I'm looking for a way to grab the camera frames to then process them with ML Kit.

[Robbo2]

I'm also looking for a way to access real-time camera frame feed to do pre-processing and live-streaming.
Anyone have suggestions?

[zoechi]

There is some work going on that looks related flutter-team-archive/plugins#679

[dustin-graham]

Thanks @zoechi, I'm working to get that PR into shape soon. It'll be a good solution for working with ML Kit.

[arloan]

I need this also. The camera plugin can only take a camera shot into image file, which is really too much time consuming for many kinds of scenario.

[dustin-graham]

@arloan I made a prototype with sending live frames from the camera plugin to another plugin. Would you mind taking a look, offering suggestions and feedback? flutter-team-archive/plugins#688

…

On Mon, Jul 30, 2018, 6:43 PM arloan ***@***.***> wrote: I need this also. The camera plugin can only take a camera shot into image file, which is really too much time consuming for many kinds of scenario. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#18968 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA6lu5FXIXsyYRL4r6XsJyJ5HjPCFTA2ks5uL7ZdgaJpZM4U-DJr> .

[arloan]

Well, I'm afraid I don't have enough knowledge of Java or Android, that's why I chose flutter ...

[Nashua354]

Maurice added the bytestream capability in the camera package
flutter-team-archive/plugins#965
and modified the firebase_ml_vision package too to make this demo
https://github.com/bparrishMines/mlkit_demo
Check it out. Its some amazing stuff.