Table of Contents |
---|
...
Note |
---|
This is a reference to the deprecated offline documentation. It needs to be replaced with new instructions on how to obtain SDK version. |
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
visage|SDK is implemented in C++ and provides C++ API which cannot be used directly in Swift without first wrapping C++ API in an NSObject and exposing it to Swift through bridging-header. Wrapping in an NSObject does not have to be one-on-one mapping with C++ classes. Instead, it can be a higher level mapping and fragments of source code from provided iOS sample projects can be used as building blocks.
A general example of how this technique is usually implemented can be found here:
https://stackoverflow.com/questions/48971931/bridging-c-code-into-my-swift-code-what-file-extensions-go-to-which-c-based-l
A way without wrapping NSObject is by wrapping C++ API in C-interface functions and exposing them through bridging-header. Example of such a wrapper is provided in visage|SDK in the form of VisageTrackerUnity Plugin which provides simpler, high-level API through a C-interface.
A general example of how this technique is usually implemented can be found here:
https://www.swiftprogrammer.info/swift_call_cpp.html
...
...
...
Important notes:
...
The package you have received is visage|SDK 8.4. The latest release is visage|SDK 8.7 but that is not available for rPI yet. If your initial tests prove interesting, we will need to discuss the possibility to build the latest version on-demand for you. visage|SDK 8.7 provides better performance and accuracy, but the API is mostly unchanged so you can run relevant initial tests.
...
visage|SDK 8.4 for rPI has been tested with rPI3b+; it should work with rPI4 but we have not tested that.
...
...
...
Which browsers does visage|SDK for HTML5 support?
The HTML5 demo page contains the list of supported browsers: https://visagetechnologies.com/demo/#supported-browsers
Please note that the current HTML5 demos have not been optimized for use in mobile browsers. Therefore, for the best results it is recommended to use a desktop browser.
Internet connection and privacy issues
Does my device/app need internet connection?
Applications developed using visage|SDK do not need an internet connection for operation.
Internet connection is needed only for license registration, which happens only once, when the application is used for the first time.
For hardware devices, this can typically be done at installation time – you would connect the device to the network, run the application once, the license registration would happen automatically, and after that no network connection is needed.
How do you handle privacy of user data? Does visage|SDK store or send it to a cloud server?
visage|SDK never transfers any user information to any server.
visage|SDK never automatically stores any user information locally.
In summary, you as the application developer have full control and responsibility for any storage or transfer of user data that you may implement in your application.
Cameras, hardware
Can I use visage|SDK with an IP camera?
Yes, visage|SDK can be used with any IP camera. However, note that visage|SDK actually does not include camera support as part of the API; the API simply takes a bitmap image. Image grabbing from camera is implemented at the application level.
Our sample projects show how to access local cameras and we currently have no ready-to-go code for accessing IP cameras. There are various ways to grab images from IP cameras, for example by using libVLC.
Usually, IP cameras should provide a URL from which you can access the raw camera feed. If you can open the URL and see the feed in VideoLAN VLC Media Player, the same URL can be used in libVLC to access the feed programmatically.
...
See also:
...
...
...
Warning |
---|
This text requires review after the introduction of new face detector in V8.7b2. Also, https://visagetechnologies.com/facetrack-features/ needs to be reviewed. |
...
...
...
...
...
...
...
...
...
...
Warning |
---|
This text requires review after the introduction of new face detector in V8.7b2. |
...
...
...
...
...
...
...
...
...
...
...
...
...
How far from a camera can a face be recognized?
This depends on camera resolution. Face Recognition works best when the size of the face in the image is at least 100 pixels.
Info |
---|
See also: |
High-level functionalities
How do I perform liveness detection with visage|SDK?
visage|SDK includes active liveness detection: the user is required to perform a simple facial gesture (smile, blink or eyebrow raising) and face tracking is then used to verify that the gesture is actually performed. You can configure which gesture(s) you want to include. As app developer you also need to take care of displaying appropriate messages to the user.
All visage|SDK packages include the API for liveness detection. However, only the visage|SDK for Windows and visage|SDK for Android contain a ready-to-run demo of Liveness Detection. So, for a quick test of the liveness detection function it would probably be the easiest to downoad visage|SDK for Windows, run “DEMO_FaceTracker2.exe” and select “Perform Liveness” from the Liveness menu.
The technical demos in Android and Windows packages of visage|SDK include the source code intended to help you integrate liveness detection into your own application.
Note |
---|
Need to insert links to the documentation of these sample projects and the the Liveness API. |
How do I perform identification of a person from a database?
This article outlines the implementation of using face recognition for identifying a person from a database of known people. It may be applied to cases such as whitelists for access control or attendance management, blacklists for alerts and similar. The main processes involved in implementing this scenario are registration and matching, as follows.
Registration
Assuming that you have an image and an ID (name, number or similar) for each person, you register each person by storing their face descriptor into a gallery (database). For each person, the process is as follows:
Locate the face in the image:
To locate the face, you can use detection (for a single image) or tracking (for a series of images from a camera feed).
See function VisageSDK::VisageTracker::track() or VisageFeaturesDetector::detectFacialFeatures().
Each of these functions returns the number of faces in the image - if there is not exactly one face you may report an error or take other action.
Furthermore, these functions return the FaceData structure for each detected face, containing the face location.
Use VisageFaceRecognition.AddDescriptor() to get the face descriptor and add it to the gallery of known faces together with the name or ID of the person.
The descriptor is an array of short integers that describes the face - similar faces will have similar descriptors.
The gallery is simply a database of face descriptors, each with attached ID.
Note that you could store the descriptors in your own database, without using the provided gallery implementation.
Save the gallery using VisageFaceRecognition.SaveGallery().
Matching
In this stage, you match a new facial image (for example, a person arriving at a gate, reception, control point or similar) against the previously stored gallery, and obtain IDs of one or more most similar persons registered in the gallery.
First, locate the face(s) in the new image.
The steps are the same as explained above in the Registration part. You obtain a FaceData structure for each located face.
Pass the FaceData to VisageFaceRecognition.ExtractDescriptor() to get the face descriptor of the person.
Pass this descriptor to VisageFaceRecognition.Recognize(), which will match it to all the descriptors you have previously stored in the gallery and return the name/ID of the most similar person (or a desired number of most similar persons);
the Recognize() function also returns a similarity value, which you may use to cut off the false positives.
Note |
---|
Need to insert links to relevant API parts in text and/or as “See also” section. |
How do I perform verification of a live face vs. ID photo?
The scenario is verification of a live face image against the image of a face from an ID. It is done in four main steps:
Locate the face in each of the two images;
Extract the face descriptor from each of the two faces;
Compare the two descriptors to obtain the similarity value;
Compare the similarity value to a chosen threshold, resulting in a match or non-match.
These steps are described here with further detail:
In each of the two images (live face and ID image), the face first needs to be located:
...
...
...
...
Note: the ID image should be cropped so that the ID is occupying most of the image (if the face on the ID is too small relative to the whole image it might not be detected).
Note |
---|
Need to insert links to relevant API parts in text and/or as “See also” section. |
How do I determine if a person is looking at the screen?
First, let me say that visage|SDK does not have an out-of-the box option to determine if the person is looking at the screen. However, it should not be too difficult to implement that. What visage|SDK does provide is:
The 3D position of the head with respect to the camera;
The 3D gaze direction vector.
Now, if you also know the size of the screen and the position of the camera with respect to the screen, you can:
Calculate the 3D position of the head with respect to your screen;
Cast a ray (line) from the 3D head position in the direction of the gaze;
Verify if this line intersects the screen.
Please also note that the estimated gaze direction may be a bit unstable (the gaze vectors appearing “shaky”) due to the difficulty of accurately locating the pupils. At the same time, the 3D head pose (head direction) is much more stable. Because people usually turn their head in the direction where they are looking, it may also be interesting to use the head pose as the approximation of the direction of gaze.
Here is a code snippet used to roughly calculate screen space gaze by combining data from visage|SDK and external information about screen (in meters) and camera relation which can be used to determine if user is looking at the screen or not (if calculated coordinates are outside the screen range):
Code Block |
---|
// formula to get screen space gaze
x = faceData->faceTranslation[2] * tan(faceData->faceRotation[1] + faceData->gazeDirection[0] + rOffsetX) / screenWidth; // rOffsetX angle of camera in relation to screen, ideally 0
y = faceData->faceTranslation[2] * tan(faceData->faceRotation[0] + faceData->gazeDirection[1] + rOffsetY) / screenHeight; // rOffsetY angle of camera in relation to screen, ideally 0
// apply head and camera offset
x += -(faceData->faceTranslation[0] + camOffsetX); // camOffsetX in meters from left edge of the screen
y += -(faceData->faceTranslation[1] + camOffsetY); // camOffsetY in meters from top edge of the screen |
Note |
---|
Need to insert links to relevant API parts in text and/or as “See also” section. |
Can images of crowds be processed?
visage|SDK can be used to locate and track faces in group/crowd images and also to perform face recognition (identity) and face analysis (age, gender, emotion estimation). Such use requires particular care related to performance issues as there may be many faces to process. Some initial guidelines:
Face tracking is limited to 20 faces (for performance reasons). To locate more faces in the image, use face detection (class FacialFeaturesDetector).
visage|SDK is capable of detecting/tracking faces whose size in the image is at least 5% of the image width (height in case of portrait images).
The default setting for the VisageFeaturesDetector is to detect faces larger than 10% of the image size and 15% in case of the VisageTracker. The default parameter for minimal face scale needs to be modified to process smaller faces.
If you are using high resolution images with many faces, so that each face is smaller than 5% of image width, one solution may be to divide the image into portions and process each portion separately; alternatively, a custom version of visage|SDK may be discussed.
For optimal performance of algorithms for face recognition and analysis (age, gender, emotion), faces should be at least 100 pixels wide.
From the previous step you have one FaceData structure for the ID image and one FaceData structure for the live image.
Pass each image with its corresponding FaceData to the function VisageFaceRecognition::extractDescriptor().
...
Pass the two descriptors to the function VisageFaceRecognition::descriptorsSimilarity() to compare the two descriptors to each other and obtain the measure of their similarity. This is a float value between 0 (no similarity) and 1 (perfect similarity).
...
If the similarity is greater than a chosen threshold, consider that the live face matches the ID face.
By choosing the threshold, you control the trade-off between False Positives and False Negatives:
If the threshold is very high, there will be virtually no False Positives, i.e. the system will never declare a correct match when in reality the live person is not the person in the ID.
However, with a very high threshold a False Negative may happen more often - not matching a person who really is the same as in the ID, resulting in an alert that will need to be handled in an appropriate way (probably requiring human intervention).
Conversely, with a very low threshold, such “false alert” will virtually never be raised, but the system may then fail to detect True Negatives - the cases when the live person really does not match the ID.
There is no “correct” threshold, because it depends on the priority of a specific application. If the priority is to avoid false alerts, threshold may be lower; if the priority is to avoid undetected non-matches then the threshold should be higher.
...
Showcase Demo
...
...
...
...
...
Visage Tracker Unity Demo
...
...
...
...
...
...
...
...
...
...
...
VisageTrackerUnityPlugin is provided with full source code within the package distribution.
I am getting EntryPointNotFoundException when attempting to run a Unity application in the Editor. Why does my Unity application does not work?
Make sure that you have followed all the steps from the documentation Building and running Unity application.
Note |
---|
Need to insert links to relevant API parts in text and/or as “See also” section. |
Verify that the build target and visage|SDK platform match. For example, running visage|SDK for Windows inside the Unity Editor and for a Standalone application will work since both are run on the same platform. Attempting to run visage|SDK for iOS inside the Unity Editor on a MacOS will output an error because iOS architecture does not match MacOS architecture.
When working with large resolution videos (>2K), video FPS significantly drops, why?
It seems that the slowdown is a result of not optimal frame mirroring (our native Windows plugin mirrors the image). Track function itself seem to be good enough. There are further optimizations that can be implemented, however for a quick fix solution try to turn off frame mirroring in Unity project by setting Mirrored property to 0 in the Unity Editor Property page for the Tracker object.
To elaborate, _grabFrame function has two working modes:
The cameraOrientation and cameraMirror values are equal to 0
The cameraOrientation and cameraMirror value are different from 0
First one will just copy pixels to another buffer, and the second one will go to the pixels manipulation process to prepare pixels for the _track() function.
Pixels manipulation is an expensive operation. To avoid this operation please turn off isMirrored value which is by default set to 1 and adjust the mirroring property in the setting of camera itself if needed.
This will result in significantly faster execution of the _grabframe() function.
Note |
---|
Important! This refers to visage|SDK 8.7.b1 and older versions. On newer versions, isMirrored value is turned off by default. |
Note |
---|
Need to insert links to relevant API parts in text and/or as “See also” section. |
Troubleshooting - Licensing Errors
The following articles describe the various error codes produced by the Visage Technologies licensing system, and what may be done to remedy each error. They are valid for visage|SDK version 8.6b1 and higher.
Error code 0x00000001 (VS_ERROR_INVALID_LICENSE)
The error you received indicates that the license is not valid. The error occurs when the BundleID is not the same as the one in your application, or when license that is registered to one product is used with another product. Please check if you’re using the correct license.
Error code 0x00000002 (VS_ERROR_EXPIRED_LICENSE)
The error you received indicates that the license has expired. Please consider renewing your license. Your Visage Technologies contact person may advise you on the options.
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
Info |
---|
[Internal - for Visage Technologies contact person] - Check on the VTLS server for the reason why license was rejected. It can be one of the following (the most common being -6):
|
...