facial-landmarks-35-adas-0002

Overview

facial-landmarks-35-adas-0002 is a neural network for regression of face landmark positions, created and released by Intel as part of the OpenVINO Model Zoo under the Apache-2.0 license.

The network takes as input a cropped RGB image of a person's face and outputs the coordinates of 35 specific facial landmark points. The full list of points is available at the model source.

This repository contains the facial-landmarks-35-adas-0002 model quantized to int8 and compiled for NXP i.MX 8M Plus, i.MX 93, i.MX 95, and i.MX 952 processors. It also contains the script used to regenerate these assets (recipe.sh) and an inference example (example.py).

Note: This is not a face detection model. Faces must be detected and cropped before being passed to this model.

Key Features

  • Lightweight CNN for 35-point facial-landmark regression
  • Small 60x60 RGB input - low memory footprint
  • int8 quantized for accelerated NPU inference on NXP i.MX platforms
  • Useful for face-centering checks, head-orientation estimation, and basic gaze estimation

Model Description

This is a quantized and compiled version of facial-landmarks-35-adas-0002 optimized for NXP i.MX 8M Plus, i.MX 93, i.MX 95, and i.MX 952 processors.

  • Base Model: facial-landmarks-35-adas-0002 (OpenVINO Model Zoo)
  • Original Model Authors: Intel Corporation
  • Original License: Apache-2.0
  • Modified by: NXP

Modifications

This model is a derivative work with the following changes from the original:

  • Format conversion: OpenVINO IR -> TensorFlow SavedModel -> TensorFlow Lite, using openvino2tensorflow.
  • Quantization: full int8 post-training quantization, uint8 input / uint8 output, input normalization data applied inside openvino2tensorflow. Because the original training data cannot be redistributed, the representative calibration dataset is ~100 images sampled from a commercially-usable ImageNetV2 subset, resized to the 60x60 model input.
  • Compilation (i.MX 93): compiled for the Ethos-U65 NPU with Vela, tags lf-6.18.20_2.0.0 and lf-6.18.37_2.1.0.
  • Compilation (i.MX 95 / i.MX 952): converted for the eIQ Neutron NPU with the eIQ Neutron SDK versions 3.1.3, 3.2.1 and 3.2.3.
  • Post-processing: the normalized [0, 1] landmark coordinates are rescaled to the original image size outside the model (see example.py).

Original model available at: facial-landmarks-35-adas-0002.

Model Information

Information Value
Input shape RGB face crop (1, 60, 60, 3), dtype uint8
Input example (Image generated on thispersondoesnotexist.com)
Output shape Face landmark positions (1, 70), uint8 quantized; dequantize with the output tensor's scale / zero point to get normalized [0, 1] coordinates. See example.py to decode it.
Output example (Image generated on thispersondoesnotexist.com)
FLOPS 41.851 MOPS
Number of parameters 4,595,372
File size (int8) 4.65 MB (assets/quantized_tflite_model/facial_landmarks_35_uint8.tflite)
Source framework Caffe / OpenVINO

Supported Hardware and Software Configurations

All models run with the eIQ LiteRT (TensorFlow Lite) runtime and the corresponding NPU delegate (VX for i.MX 8M Plus, Ethos-U for i.MX 93, Neutron for i.MX 95 / i.MX 952). The same information is captured in release_manifest.json.

Asset Runtime Precision HW target BSP & SDK versions Link to asset
i.MX 8M Plus Quantized LiteRT INT8 i.MX 8M Plus - Download
i.MX 93 Precompiled LiteRT INT8 i.MX 93 BSP lf-6.18.20_2.0.0, VELA_TAG = lf-6.18.20_2.0.0 Download
i.MX 93 Precompiled LiteRT INT8 i.MX 93 BSP lf-6.18.37_2.1.0, VELA_TAG = lf-6.18.37_2.1.0 Download
i.MX 95 Precompiled LiteRT INT8 i.MX 95 BSP lf-6.18.20_2.0.0, Neutron SDK v3.1.3 Download
i.MX 95 Precompiled LiteRT INT8 i.MX 95 BSP lf-6.18.37_2.1.0, Neutron SDK v3.2.1 Download
i.MX 95 Precompiled LiteRT INT8 i.MX 95 BSP lf-6.18.37_2.1.0, Neutron SDK v3.2.3 Download
i.MX 952 Precompiled LiteRT INT8 i.MX 952 BSP lf-6.18.20_2.0.0, Neutron SDK v3.1.3 Download
i.MX 952 Precompiled LiteRT INT8 i.MX 952 BSP lf-6.18.37_2.1.0, Neutron SDK v3.2.1 Download
i.MX 952 Precompiled LiteRT INT8 i.MX 952 BSP lf-6.18.37_2.1.0, Neutron SDK v3.2.3 Download

The hardware-independent int8 model is also provided for host (x86) inference and as the input to the compilers above: assets/quantized_tflite_model/facial_landmarks_35_uint8.tflite.

Limitations and Biases

This model inherits all limitations from the original facial-landmarks-35-adas-0002 model. Additional limitations:

  • Task scope: This model only regresses 35 facial landmark coordinates on an already-cropped face image. It does not perform face detection, face recognition, or attribute estimation, and faces must be detected and cropped upstream.
  • Dataset bias: According to the model source, the model was trained on an internal dataset of 1000 images of 300 different people with various facial expressions. Since this training dataset is not publicly available, the accuracy of the final quantized model could not be independently verified, and any demographic or pose biases present in that dataset cannot be assessed. This model should not be used in critical scenarios.

Training

According to the model source, the original model was trained by Intel on an internal dataset of 1000 images of 300 different people with various facial expressions. NXP did not retrain this model; only format conversion, quantization and compilation were performed (see Modifications).

Download and Run

The recipe installs the eIQ Neutron SDK from the NXP eIQ PyPI Distribution Portal. Configure pip once to use it:

python -m pip config --user set global.index https://eiq.nxp.com/repository
python -m pip config --user set global.index-url https://eiq.nxp.com/repository
python -m pip config --user set global.trusted-host eiq.nxp.com
python -m pip config --user set global.extra-index-url https://pypi.org/simple
python -m pip config --user set global.find-links https://eiq.nxp.com/repository/

To recreate the int8 TFLite model, point DATASET_PATH at a local ImageNetV2 subset and run:

DATASET_PATH=/path/to/imagenetv2 bash recipe.sh

The Vela tag and eIQ Neutron SDK version can be passed as positional arguments (defaulting to lf-6.18.20_2.0.0 and 3.2.3 respectively):

DATASET_PATH=/path/to/imagenetv2 bash recipe.sh lf-6.18.37_2.1.0 3.2.3

This produces:

  • assets/quantized_tflite_model/facial_landmarks_35_uint8.tflite - hardware-independent int8 model (also used for i.MX 8M Plus)
  • assets/imx93/<VELA_TAG>/facial_landmarks_35_uint8_vela.tflite - Vela-compiled for i.MX 93
  • assets/imx95/<NEUTRON_SDK_VERSION>/facial_landmarks_35_uint8_converted.tflite - Neutron-converted for i.MX 95
  • assets/imx952/<NEUTRON_SDK_VERSION>/facial_landmarks_35_uint8_converted.tflite - Neutron-converted for i.MX 952

An example of how to use the model is in example.py (run on a PC), using its -m/--model, -i/--input and -o/--output flags:

# Run on PC:
python example.py -m assets/quantized_tflite_model/facial_landmarks_35_uint8.tflite -i example_input.jpg -o example_output.jpg

Please refer to the i.MX Machine Learning User Guide for details on how to run the models on each target platform.

License

This model, including the quantized and compiled assets and the scripts in this repository, is released under the Apache License 2.0 (see LICENSE). The original facial-landmarks-35-adas-0002 model is also released under the Apache License 2.0 (see MODEL_LICENSE).

Citation

If you use this model, please cite the original model source:

@misc{facial-landmarks-35-adas-0002,
  title = {facial-landmarks-35-adas-0002},
  author = {{Intel Corporation}},
  howpublished = {OpenVINO Open Model Zoo},
  url = {https://docs.openvino.ai/2022.3/omz_models_model_facial_landmarks_35_adas_0002.html}
}
Downloads last month
45
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Collections including nxp/facial-landmarks-35-adas-0002-imx