Google ML Kit: For On-device Machine Learning

Machine Learning in Artificial Intelligence

ML Kit Pose Detection

Artificial intelligence is the ability of computational systems to perform tasks that require human intelligence otherwise. The early models of AI were hardcoded algorithms that responded to a finite set of input triggers with predictable outputs such as playing checkers. Later, when the application of AI extended to diverse domains and complex use cases, it became a necessity for the system to adapt and change with new trigger sets. This paved the way to the new sub-field, Machine Learning.

Google ML Kit

In June 2020 Google deprecated two of its existing solutions – on-device APIs provided through ML Kit for Firebase SDK and the Google Mobile Vision SDK in favor of the ML Kit mobile SDK. This new SDK was in beta version since 2018 and was made Generally Available (GA) 2 years later in 2020. ML Kit is free of cost machine learning solution for iOS and Android apps, focusing on ‘on-device’ ML APIs. In addition, it provides an easy interface to additional google cloud AI solutions such as Google Cloud Vision API, TensorFlow Lite, and Neural Network APIs.

Advantages of ML Kit Over Cloud-based Solutions

ML Kit processes the data on the mobile device, unlike other solutions which send it over to a server, process it, and return the results which are then consumed by the application. On-device data processing results in the following advantages.

  • Latency
  • Data processing is real-time without any network latency
  • Offline capability
  • On the device, APIs eliminate the dependency on network connectivity or bandwidth
  • Data security
  • Data need not be sent over the wire for the processing which eliminates the need for in-transit data security considerations

On-device API support

On-device APIs supported by ML Kit can be broadly classified into three categories. Among these, all APIs except Selfie Segmentation, Pose Detection, and Entity Extraction are in General Availability while these three are in Beta.

  • Vision
    • Barcode scanning
    • Pose detection
    • Face detection
    • Image classification and labelling
    • Object detection and tracking
    • Selfie segmentation
    • Text recognition
  • Natural Language Processing
    • Language identification
    • Language translation
    • Smart Reply
    • Entity Extraction
  • Others
    • Digital ink recognition or handwriting recognition

Machine Learning Models and Training

Google ML Kit uses Google-trained ML models by default. These are well-trained models that cover a wide range of applications and are sufficient for general use cases. ML Kit also supports custom ML models for users targeting specific use cases which are not satisfied by the default google models. Users can choose a pre-trained TensorFlow Lite model which is already published through TensorFlow Hub or even train their own TensorFlow Lite models to meet the requirements.

ML Kit Integration

iOS and android ML Kit libraries are published as dynamic dependencies.

  • ML Kit libraries for iOS are distributed as cocoapods. ML Kit integration requires Xcode 12.5.1 or higher and it does not support 32-bit architectures (i386 and armv7)
  • Gradle dependencies of ML Kit for Android use the namespace ‘’. ML Kit Android SDK has back support from Android 4.0 and hence will work on older versions of android. It does not require any common libraries as in the case of firebase SDK and each API is supported through a single dependency


Author Details

Mathews Philip

Mathews Philip is a Technology Lead at Digital Experience in Infosys. He is specialised on iOS in Mobile Technologies and on Angular in Front End Stack. He is part of the Energy, Utilities, Communications and Services unit in Infosys and helps in delivering digital transformation for organisations across the globe. He will be focusing on the emerging technologies, mainly Artificial Intelligence.

Leave a Comment

Your email address will not be published.