VisageFaceAnalyser now offers multi-frame analysis functionality which performs estimations on the selected high-quality frames and returns an averaged, smoothed value. This functionality is available via the VisageFaceAnalyser::analyseStream() function.

Introducing smaller, faster and more accurate age and gender estimation models which completely replace the old models that will no longer be distributed.

C++ API:

VisageSDK namespace

Introducing new FaceData class member for getting the face bounding box:

• FaceData::faceBoundingBox

C# API:

VisageCSWrapper namespace

Introducing new FaceData class member for getting the face bounding box:

• FaceData.FaceBoundingBox

C++ API:

VisageFaceAnalyser API was refactored to optimise usage and to include newly introduced multi-frame analysis feature.

Old face analysis functions were removed:

• VisageFaceAnalyser::estimateAge()

• VisageFaceAnalyser::estimateGender()

• VisageFaceAnalyser::estimateEmotion()

and replaced with new ones:

• VFAReturnCode VisageFaceAnalyser::analyseImage(const VsImage* image, const FaceData& faceData, const int options, AnalysisData& results)

• VFAReturnCode VisageFaceAnalyser::analyseStream(const VsImage* frame, const FaceData& faceData, const int options, AnalysisData& results, const int faceIndex = 0)

• void VisageFaceAnalyser::resetStreamAnalysis(const int faceIndex = -1)

Three separate calls to old analysis estimation functions can now be performed by a single call to analyseImage() or analyseStream() where parameter options specifies which analysis operations to perform.

For more details on the updated API please see VisageFaceAnalyser API documentation.

Introducing new AnalysisData structure which contains face analysis results (age, gender and emotions) returned by the VisageFaceAnalyser::analyseImage() and VisageFaceAnalyser::analyseStream() functions.

C# API:

Due to native VisageFaceAnalyser API changes, VisageCSWrapper's VisageFaceAnalyser API was updated accordingly.
New AnalysisData structure which contains face analysis results (age, gender and emotions) was added as well.

C++ API:

Prototype of VisageFeaturesDetector constructor and the following method has been changed:

• VisageFeaturesDetector()

  to:

  VisageFeaturesDetector(const char* detectorConfigFile)

• bool VisageFeaturesDetector::Initialize(const char* path)

  to:

  bool VisageFeaturesDetector::Initialize()

For more details please see VisageFeaturesDetector API documentation.

C# API:

Prototype of VisageFeaturesDetector constructor has been changed:

• VisageFeaturesDetector()

  to:

  VisageFeaturesDetector(string detectorConfigFile)

Following method has been removed:

• bool VisageFeaturesDetector::Initialize(String path)

For more details please see VisageFeaturesDetector API documentation.

C++ API:

Introducing new functions for getting and setting specific data paths required by tracking algorithm:

• const string& getVftDataPath(void) const

• const string& getVfdDataPath(void) const

• const string& getPrDataPath(void) const

• const string& getErDataPath(void) const

• bool setVftDataPath(string vft_data_path)

• bool setVfdDataPath(string vfd_data_path)

• bool setPrDataPath(string pr_data_path)

• bool setErDataPath(string er_data_path)

Removed functions for getting and setting the obsolete data path no longer required by tracking algorithm:

• const string& getBdtsDataPath() const
  bool setBdtsDataPath(string bdts_data_path)

C# API:

VisageConfiguration class contains new public attributes:

• VisageConfiguration.VftDataPath

• VisageConfiguration.VfdDataPath

• VisageConfiguration.PrDataPath

• VisageConfiguration.ErDataPath

Removed public attribute:

• VisageConfiguration.BdtsDataPath

Removed

• Old tracking algorithm data (vft/fa) - landmarks.bin and init_shape.bin

• Old gender estimation data (vfa/gd) - gd.lbf

• Old data path parameter - bdts_data_path parameter

Added

• New tracking algorithm data (vft/fa) - aux_file.bin

• New gender estimation data (vfa/gd) - gd.vino.bin and gd.vino.xml

• New data path parameters:
• vft_data_path - path to the folder tracking algorithm data files required by tracker
• vfd_data_path - path to the folder containing algorithm data file required by tracker and detector
• pr_data_path - path to the folder containing containing pupils' refinement data files
• er_data_path - path to the folder containing ears' refinement data files

Configuration files using the removed

 bdts_data_path 

parameter should be updated to use new data paths instead.

Modified

• Age estimation data (vfa/ad) - ae.vino.bin and ae.vino.xml
  
• Tracking algorithm data (vft/fa) - d1qy.vino.bin, d1qy.vino.xml, d2.vino.bin and d2.vino.xml

Projects using older versions of these files should be updated to contain the newest data files from the data folder.

The default legacy algorithm that could be allowed via the use_vnn configuration parameter (value 0) has been removed. The VNN algorithm is now the default and only tracking and detection algorithm.

The algorithm can be configured to improve feature points precision and robustness, over tracking speed (performance) by setting the refine_landmarks parameter to 1 in the configuration file. Otherwise, when performance is preferred over feature points precision, the recommendation is to disable refine_landmarks parameter by setting it to 0 in the configuration file.

Face tracking and detection algorithms are enhanced so that they can track and detect faces wearing protective masks of various colors and patterns.

Visible contour landmarks from group 13 (13.1-13.17) are no longer available (not detected or estimated). Instead, contour landmarks are now provided only in the form of a physical contour within group 15 (15.1-15.17). For more details please see FDP documentation.

Introducing new C++ and C# API for getting SDK version: 

String^ Get(void)

Parameter *frame is now mandatory for the following method: 

VisageLivenessBlink::update(const FaceData *faceData, VsImage *frame)

Introducing new FaceData class member for getting the rotation of the face from the camera viewpoint: 

FaceData::faceRotationApparent

FaceData::faceRotationApparent

Facial Features Tracker - High.cfg to Facial Features Tracker.cfg
Facial Features Tracker - High - With Ears.cfg to Facial Features Tracker - With Ears.cfg

refine_landmarks parameter

As a consequence of improving algorithms, removing legacy algorithm, there are certain changes in the data files.

Samples displaying feature points are updated to draw physical contour points.

The in-house developed runner is no longer available and is being replaced by OpenVINO™ toolkit, which is now the only and default neural network runner.

It is implemented in the following libraries:

• OVPlugin.dll,

• inference_engine.dll,

• MKLDNNPlugin.dll,

• mkl_tiny_tbb.dll,

• tbb.dll

• libmmd.dll,

• svml_dispmd.dll

OpenVINO is a trademark of Intel Corporation or its subsidiaries.

The old face recognition model is being replaced by a smaller, faster and more accurate face recognition model.

Introducing a more accurate and robust face detection model. The new model is used in face tracking and face detection when use_vnn configuration parameter is set to 1. Otherwise, the previous face detection model will be used.

VNN algorithm can now be configured for better performance, at the cost of feature points precision. This mode is enabled by setting use_vnn parameter in configuration file to 1.

Added support for multiple image format for extractDescriptor() and addDescriptor()methods.

use_vnn configuration parameter added to VisageFeaturesDetector configuration file FaceDetector.cfg(located in Samples/data/bdstdata) and set to value 1.

use_vnn configuration parameter values changed and additional value added:

As a consequence of improving algorithms, removing in-house developed runner, and improving VNN algorithm, there are certain changes in the data files.

VisageTrackerUnityPlugin

Function _grabFrame() captures RGB image, instead of BGR image.

VisageTrackerUnityDemo

The previously used BGRATex texture shader for applying captured frame image data was replaced with RGBATex texture shader (Tracker.cs script).

To improve the performance of our algorithms and to support a wider variety of neural network models, we are introducing a configurable framework for choosing between different neural network runners.

As a result, additional configuration file NeuralNet.cfg is now included in visage|SDK (located in Samples/data). This file allows the users to configure which runner will be used by visage|SDK. Users can choose between:

• Visage Technologies' runner developed in-house and

• OpenVINO™ toolkit.

For more information on the parameters of NeuralNet.cfg file see the API page.

OpenVINO is a trademark of Intel Corporation or its subsidiaries.

New experimental algorithm for face tracking and alignment introduced - VNN algorithm.

For the price of slightly reduced tracking speed/performance, it significantly improves tracking quality and robustness. VisageTracker and VisageFeaturesDetector can be configured to use VNN algorithm via configuration parameter - use_vnn.

Introducing new C++ and C# API for programmatically changing VisageTracker configuration parameters, including new additional classes and templates:

The aforementioned classes are used in conjunction with new VisageTracker methods:

FDP group 10 (ears) has been extended from 10 to 24 points (12 points per ear) as part of the ear tracking feature.

VisageTracker method stop has been deprecated from both APIs.

Prototype of method

void initializeLicenseManager(const char *licenseKeyFileName)

changed to

int initializeLicenseManager(const char *licenseKeyFileName)

FeaturePoint class and FDP class have additional property and functions, respectively to conform with the C++ API.

Two additional configuration files have been added. One for the ear tracking feature and one for the novel tracking algorithm.

• refine_ears parameter added, off (0) by default. Toggles the tracking and refinement of ear points (group 10) for VisageTracker and VisageFeaturesDetector.

• use_vnn parameter added, off (0) by default. Toggles usage of the new experimental algorithm for face tracking and alignment - VNN algorithm

A new model file has been added for ear tracking functionality - jk_300_wEars.wfm. The model contains an additional 334 polygons and its vertices are mapped to 14 new FDP points in group 10 (10.11 - 10.24).

New data files and model files for ear tracking added

New data folder and data files added for VNN algorithm in Samples/data/bdtsdata/NN/vnn

FacialAnimationUnityDemo sample application has been removed and will no longer be distributed.

VisageTrackerUnityDemo sample applicationisdistributed as a Unity project, not as a prebuilt application.

Build and run instructions are provided in the VisageTrackerUnityDemo sample documentation.

• smoothing_factors parameter
  Due to the re-implementation of the smoothing algorithm in VisageTracker, default values and optimal ranges for this parameter have been changed in all configurations.

VisageRendering.cs:

Methods DisplayEmotion() and DisplayAgeGenderName() have changed the prototypes from:

public void DisplayEmotion(FaceData trackingData, float[] emotions, int width, int height,
                           bool face_recognition_mode=false)
public void DisplayAgeGenderName(FaceData trackingData, float age, int gender, string name, 
                                 int width, int height, int drawingOptions, 
                                 bool face_recognition_mode=false)

to:

public void DisplayEmotion(FaceData trackingData, float[] emotions, int width, int height, 
                           bool display_background = false)
public void DisplayAgeGenderName(FaceData trackingData, float age, int gender, string name, 
                                 int width, int height, bool display_background = true)