ML Kit encompasses all the existing Google offerings for machine learning on mobile. It bundles the Google Cloud Vision API, TensorFlow Lite, and the Android Neural Networks API together in a single SDK, as shown:
ML Kit enables developers to utilize machine learning in their mobile applications for both Android and iOS apps, in a very easy way. Inference can be carried out by invoking APIs that are either on-device or on-cloud.
The advantages of on-device APIs are that they work completely offline, and are more secure as no data is sent to the cloud. By contrast, on-cloud APIs do require network connectivity, and do send data off-device, but allow for greater accuracy.
ML Kit offers APIs covering the following machine learning scenarios that may be required by mobile application developers:
- Image labeling
- Text recognition
- Landmark detection
- Face detection
- Barcode scanning
All these APIs are implemented using complex machine learning algorithms. However, those details are wrapped. The mobile developer need not get into the details of which algorithms are used for implementing these APIs; all that needs to be done is to pass the desired data to the SDK, and in return the correct output will be received back, depending on which part of ML Kit is being used.
If the provided APIs don't cover a specific use case, you can build your own TensorFlow Lite model. ML Kit will help to host that model, and serve it to your mobile application.
Since Firebase ML Kit provides both on-device and on-cloud capabilities, developers can come up with innovative solutions to leverage either or both, based on the specific problem at hand. All they need to know is that on-device APIs are fast and work offline, while Cloud APIs utilize the Google Cloud platform to provide predictions with increased levels of accuracy.
The following diagram describes the issues to consider when deciding between on-device or on-cloud APIs:
