Image Recommender

Details

Image recommender is a custom-built application that searches similar photos of a given photo, based on the artistic style of the photo, as well as the content present in that photo. It is similar with what Google image search does, but for visual arts and creative industries. The Image Recommender can be used in any field where similar pictures need to be identified from a known set. The system uses an approach based on Convolutional Neural Networks (CNN). The image recommender is our state-of-the art technology in the field of image recognition.

At its core, the image recommender system runs in two phases:

model training & tuning and feature extraction, performed offline
image query & retrieval, performed online

Features are learned in three multi-layer CNNs (art-style, content, style) and then combined in one large feature-vector (meta-descriptor) which is further used for computing the image similarities

the ‘content’ model is the pre-trained GoogLeNet model described by Szegedy et al. in ILSVRC 2014 and learns 1000 features describing the content of the images
the ‘art-style’ model is a tuned version of the GoogLeNet model, trained on 85K paintings annotated with 25 style/genre labels
the ‘style’ model is the pre-trained ‘Flickr Style’ model described by Karayev et al. and outputs 20 features
a 4th model is used to learn the type of the images (art, content or style) and its output is used as a dynamically feature weighting mechanism

Behnid the scenes

The system is designed with extensibility in mind. It has a core that aggregates multiple pluggable modules: style, art-style, content, color
For system scalability consideration we run multiple instances of each pluggable module.
The frontend has an adaptive/responsive layout using Bootstrap in order to address all kind of devices (including mobile)
For performance and scalability considerations, feature learning and extraction are performed on the GPU, using the CUDA software architecture provided by Nvidia.
Training and feature extraction are done using the Caffe framework, because of its speed and modularity.

Would you like to know more about this project? Reach out today!