Flutter – On-Device Machine Learning using ML Kit
Machine Learning is being used widely in projects today. It requires knowledge of Machine Learning to create models, for application developers who want to use Machine Learning in the project but don’t know how to create a model. In Flutter, certain developments have been done by Google to help developers create the application they want. Although there is a package google_ml_kit however it is only available for android yet. In this article, we will use ML Kit.
ML Kit is created by Google for developers to build mobile applications that involve machine learning easily. It can be used for text recognition, face and pose detection, barcode scanning, etc. We are going to create an application that detects items in the Image and labels them.
Firstly, let us see the application we are going to create in this tutorial –
- Capture the Image
- Preprocess Image
- Identify items with label, index, and confidence.
Install the dependency:
Include the google_ml_kit package in pubspec.yaml file of the app, in order to use ML Kit features by Google.
To use the camera feature we first need to create a screen that will capture the image. To include the camera feature we need to use the camera library of Flutter.
Configure both libraries by running pub get.
Import the camera library in camera_screen.dart, and initialize the CameraController. We are also finding the number of cameras available and selecting the first camera from the option to take pictures.
Call the method inside initState(), to start the camera when the app starts:
Before moving to the next screen we need to dispose of the _controller to avoid any memory leak:
Now, to take pictures, we are creating another function _takePicture(), when the button will be clicked to capture an image this function will be invoked. It will check whether the CameraController is initialized or not. It will return the ImagePath where the image will be stored in the device after capturing. If any error occurs, it will throw an exception. After capturing the image, it will redirect us to another screen, detail_screen.dart. So, create another file detail_screen.dart to show the results.
Full Code for camera_screen.dart
Work on detail_screen.dart:
First, we need to process the image, get imageSize to be displayed on the screen.
Initialize imageLabeler from GoogleMLKit in the class.
Now, we will create another function _recognizeImage(), first, we will get image size then get the image file. We will store all the image labels in list labels of type List<ImageLabel>. Then, through for loop, we will retrieve label, index, and confidence for each Image label and store them in their respective lists – imagesData, indexData, and confidenceData.
Now, it’s time to display results on the screen.
Full Code for detail_screen.dart:
Now, call CameraScreen in main.dart.