Image Recognition module

The Magnolia Image Recognition module searches for untagged images, recognizes and tags them via either a local pre-trained neural network or an external image recognition solution. The module also provides the integration of the image recognition feature with content tags.

Module structure

Installing

Maven is the easiest way to install the modules. Add the following dependencies  to your webapp. All the other necessary dependencies will be brought in automatically:

<dependency>
  <groupId>info.magnolia.ai.image</groupId>
  <artifactId>magnolia-image-recognition</artifactId>
  <version>1.1.1</version>
</dependency>

<dependency>
  <groupId>info.magnolia.ai.image</groupId>
  <artifactId>magnolia-content-tags-integration</artifactId>
  <version>1.1.1</version>
</dependency>

<dependency>
  <groupId>info.magnolia.ai.image</groupId>
  <artifactId>magnolia-image-recognition-api</artifactId>
  <version>1.1.1</version>
</dependency>

<dependency>
  <groupId>info.magnolia.ai.image</groupId>
  <artifactId>magnolia-amazon-image-recognition</artifactId>
  <version>1.1.1</version>
</dependency>

<dependency>
  <groupId>info.magnolia.ai.image</groupId>
  <artifactId>magnolia-local-image-recognition</artifactId>
  <version>1.1.1</version>
</dependency>

• magnolia-content-tags-integration.jar
• magnolia-image-recognition-api.jar
• magnolia-image-recognition.jar
• magnolia-local-image-recognition.jar
• magnolia-amazon-image-recognition.jar

AWS libraries

For Amazon Rekognition, you also need the latest version of the following JARs:

• http://central.maven.org/maven2/com/amazonaws/aws-java-sdk-core/

• http://central.maven.org/maven2/com/amazonaws/aws-java-sdk-rekognition/

General configuration

In the module configuration you:

• Choose between the local image recognition service and Amazon Rekognition service.
• Adjust the minConfidence property.
• Fine-tune the performance of parallel image recognition.

Choosing a service provider

Choose which image recognition solution you want to use: the local image recognition service provided by Magnolia or the Amazon Rekognition service. 

You can configure the service provider in /image-recognition/config.yaml.

Example:

currentService: info.magnolia.ai.image.local.LocalImageRecogniser

If you want to use the Amazon Rekognition service, you must change the currentService property and see the Amazon-specific configuration below.

Recognition confidence

For the Amazon Rekognition service, see Amazon Rekognition configuration options.

For the local image recognizer, set it with the minConfidence property under /local-image-recognition/config.yaml.

Parallel recognition threads

Image recognition can be a very intensive process in terms of CPU resources. One of the functions of our implementation of image recognition is that the recognition process can be delegated to several parallel threads. With more threads working you can obtain the results faster.

You can specify the number of parallel  image recognition threads for  the image recognition service provided by the  image-recognition-api  submodule. Set the number in the  magnolia.properties  configuration file through a Magnolia property called  magnolia.image.recognition.numberOfThreads . 

For example, magnolia.image.recognition.numberOfThreads=3 enables three recognition threads. If the property is not set in the file, the default number of threads used by the system is one.

Amazon Rekognition service configuration

Prerequisites

This section lists the prerequisites for using Magnolia with Amazon Rekognition Image recognition service.

AWS service permissions

First, make sure that you have acquired appropriate permissions for the service in the Amazon IAM Management Console:

Access keys

You need an AWS secret access key to access the service. Access keys consist of two parts: 

• Access Key ID 
• Secret Access Key

Generate the key in the security credentials section of the Amazon IAM Management Console.  (In the navigation bar on the upper right, choose your user name, and then choose My Security Credentials.)

Add the two parts of the key to your Magnolia instance in the Password manager app using the following names:

For more information about the keys, see Understanding and Getting Your Security Credentials.

Region name

You need to know a region name to configure the Amazon Rekognition Image service in Magnolia. To reduce data latency, AWS offers several regional endpoints. Each of the endpoints can be referred to in service configurations by a region name, for example eu-west-1.

For more information see Amazon's AWS Regions and Endpoints  page. Please note that if you pick a region that does not support this service, you may get erratic results.

Configuration options

Under /amazon-image-recognition/config.yaml, you must configure the following properties for the recognition service: 

region:
  name: your_aws_region_name
maxLabels: 10
minConfidence: 50
supportedFormats:
  png: png
  jpg: jpg
  jpeg: jpeg

Properties

Using other image recognition solutions

You can implement another image recognition solution or integrate your app with another third-party image recognition solution using the info.magnolia.ai.image.ImageRecogniser interface provided by the image-recognition-api submodule:

    /**
     * Takes image bytes as parameter and returns a {@link Collection collection}
     * of {@link ImageLabel Image label}s as output.
     *
     * <p>
     *     Returns empty collection for the cases below:
     *     <li>Upon exception</li>
     *     <li>Image couldn't be recognised</li>
     * </p>
     */
    Collection<ImageLabel> recognise(byte[] imageBytes);

Recognizing images

The image recognition and tagging action is executed during the startup of the author instance and every time you upload a new image asset.

You can also trigger the action manually in the Assets app by selecting one or more assets and clicking the Run recognition action. 

Images that have already been tagged are marked as such using a JCR property called lastTaggingAttemptDate. Executing the manual Run recognition action forces a new tag to be set even if the image was previously tagged.  

Removing tags

Once an image has been tagged, you can remove some or all of the tags by selecting the asset and clicking the Modify tags action in the Assets app.

In the dialog box that opens, you can remove individual tags or click Remove all tags.