[Software] Saliency Map Algorithm : MATLAB Source Code

Below is MATLAB code which computes a salience/saliency map for an image or image sequence/video (either Graph-Based Visual Saliency (GBVS) or the standard Itti, Koch, Niebur PAMI 1998 saliency map). See the included readme file for details. I also have a newer, simpler version implementing only the Itti algorithm (see simpsal/readme.txt). Additionally, there is also code to compute a saliency map based on the "Image Signature" as described in this PAMI paper by Xiaodi Hou, Jonathan Harel, and Christof Koch.

Please email me if you have any questions!

Last updated July 24, 2012.

Download:

Please select one of the following to download:

  1. [gbvs.zip]
    This package includes an implementation of the full GBVS algorithm. It also lets you compute an Itti, Koch, Niebur saliency map. It includes a function for computing the ROC score between eye-movements (provided as x,y fixation points) and a saliency map, and a function for displaying a saliency map overlayed on top of an image (as in the screenshot below).

    Right after you download the zip file, you must change into the gbvs/ directory and run:

    >> gbvs_install

    You only need to run that the first time. Afterwards, you can generate a saliency map as follows:

    To load an image:

    >> img = imread('samplepics/1.jpg');

    To compute a GBVS map:

    >> map = gbvs(img); % map.master_map contains the actual saliency map

    To compute an Itti-Koch-Niebur map:

    >> map_itti = ittikochmap(img); % map_itti.master_map contains the actual saliency map

    And you can visualize the saliency map on top of your image as follows:

    >> show_imgnmap( img , map );

    Note: If you install GBVS, make sure you put it in a directory the full path to which does not contain spaces.

  2. [simpsal.zip]
    This is a radically simplified version of the code implementing only the Itti algorithm. See simpsal/readme.txt for details. This is useful if you want to edit something minimal for research purposes. This is also the preferred implementation if you are working with video data. More features are included in the GBVS package above.

    Note 1: if you want the output saliency map to be the same size as the input image, call simpsal like this:

    >> map = simpsal(img);
    >> mapbig = mat2gray(imresize( map , [ size(img,1) size(img,2) ] ));


    Note 2: If you are looking for a very quick & dirty saliency map, call simpsal with default_fast_param (e.g., simpsal(img,default_fast_param)).

    The output of simpsal is not identical to that output by the Itti algorithm option from the GBVS package. simpsal is a bit faster and more accurate (in fixation predictions) than the Itti implementation above. Surprisingly, the first one is a more faithful reproduction of the original algorithm (exact mechanism of center surround subtraction and scale parameters and selection). However, they are identical in spirit and structure and vary only slightly in output. Graph-Based Visual Saliency is more accurate than both (see below).

  3. [signatureSal.zip]
    This package includes an implementation of the Image Signature Saliency algorithm, as described in Xiaodi Hou, Jonathan Harel, Christof Koch: Image Signature: Highlighting Sparse Salient Regions. IEEE Trans. Pattern Anal. Mach. Intell. 34(1): 194-201 (2012)

    >> map = signatureSal(img);

Note: If you have trouble/run into an error running imshow, imresize, prctile, or mat2gray, try downloading stupidtools.zip and put the contents in the directory containing the saliency code.

Copyright:

The Itti, Koch, Niebur algorithm is provided for educational use only and for copyright information on that please read this.
The GBVS algorithm can be used for any educational or research purposes, but only if it is fully cited/acknowledged.

Reference:

    See the original paper by Jonathan Harel. A poster is also available.

    Please cite use of this code as:
    - J. Harel, C. Koch, and P. Perona, "Graph-Based Visual Saliency", Proceedings of Neural Information Processing Systems (NIPS), 2006. [pdf]
    - and/or -
    - J. Harel, A Saliency Implementation in MATLAB: http://www.klab.caltech.edu/~harel/share/gbvs.php
    - or (if appropriate) -
    - X. Hou, J. Harel, and C. Koch, "Image Signature: Highlighting Sparse Salient Regions." IEEE Trans. Pattern Anal. Mach. Intell. 34(1): 194-201 (2012) [pdf]
Feel free to contact me via email with any questions.

Screen Shot:



My Interpretation of why GBVS is more predictive than Itti:

Two very trivial factors which greatly affect the measured prediction accuracy of saliency algorithms (up to 30% improvement in some cases) are (1) level of final blur (for instance, the width of a Gaussian blur kernel applied to the master map), and (2) extent of center bias (for instance, a global center bias multiplicatively applied to the master map). So, inherent variability in these two contributed to the reported performance gain the NIPS 2006 paper. However, even after controlling for these, there is a performance difference between GBVS and Itti. For instance, on a database of 200 face images you will find 0.841 mean ROC for GBVS and 0.812 mean ROC for Itti (NSS values 1.321 and 1.228 resp.) after optimizing for blur and center bias. Based on an inspection of many saliency maps, I believe that this remaining performance gain is due to long-range competition in the GBVS algorithm which is not captured accurately enough by the Itti algorithm.

-JH