DeepGaze II

DeepGaze II is a saliency model for predicting fixations in images. It is developed by Matthias Kümmerer at the bethge lab at the Centre for Integrative Neursciene at the University of Tübingen and is currently (October 2016) highest ranking in the MIT Saliency Benchmark on the MIT300 dataset (AUC, sAUC). ICF (Intensity Contrast Features) is a model using the same architecture but with access to only simple low level intensity contrast features. It is currently the best model for saliency prediction without the use of transfered deep fetures.

This webpage provides a service for scientists to hand in images of their own interest and be provided with the predictions of DeepGaze II and ICF. As of now, there is no means to hand in a large set of images at once. If you do need preditions for a larger number of images, feel free to contact us via mail. Also, we are happy to collaborate if you have specific needs for your research.

You can download both the DeepGaze II model and the ICF model as tensorflow models together with an Jupyter notebook demonstrating their use

Submit an image

You can find all details about DeepGaze I, DeepGaze II and ICF in the following papers:

  • Matthias Kümmerer, Thomas S.A. Wallis, Matthias Bethge: Understanding Low- and High-Level Contributions to Fixation Prediction ICCV 2017
  • Matthias Kümmerer, Thomas S.A. Wallis, Matthias Bethge: Saliency Benchmarking: Separating Models, Maps and Metrics arXiv:1704.08615
  • Matthias Kümmerer, Lucas Theis, Matthias Bethge: Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNet (ICLR 2015 workshop paper)

Changelog

  • Dec 15th 2017: Models available for download
  • Oct 22th 2017: Added ICF model
  • May 26th 2017: New saliency map types according to arXiv:1704.08615
  • Oct 17th 2016: Added sampling visualizations
  • Oct 6th 2016: First release