1. Follow the parameters setting in the paper and the training prototxt provided by kevinmcguinness in issue #3, i train the deep net based on VGG_CNN_M, but the net does not convergent. My training prototxt is: net:"train_deep_sal.prototxt" test_iter: 500 test_interval: 1000 display: 10 average_loss: 20 lr_policy: "step" gamma: 0.5 stepsize: 100 base_lr: 0.00000013 momentum: 0.9 iter_size: 1 max_iter: 24000 weight_decay: 0.0005 snapshot: 5000 snapshot_prefix: "train_deep_SalNet" solver_mode: GPU test_initialization: false Is it right?
2. In the training prototxt, saliency mean is 31 and scale is 2/255, can it resize the saliency map to [-1, 1]? It seems that ([0, 255]-31)*(2/255) dose not match [-1 1], and this will mark more regions salient?
3. In the post-process stage, the net_output add 127 but why not 31?

Hi, i am not very familier with the python code i might be doing something wrong, but i'm receiving same output for each image with the (modified) demo code. file names and indexes are fine, but the output gives same results for the images.

def saliencyPredictor(xt, net2, url, outUrl): y_pred_test = net2.predict(xt) i=0 for file in glob.glob(url+"*.png"):
img = misc.imread(file)
tmp = y_pred_test[i].reshape(48,48) blured= ndimage.gaussian_filter(tmp, sigma=3) y = misc.imresize(blured,(img.shape[0],img.shape[1]))/255. #misc.imsave(outUrl + os.path.basename(file), y) base = os.path.basename(file) scipy.io.savemat(outUrl + os.path.splitext(base)[0] + ".mat", {'y':y}); i= i+1

Are you planning to share your caffe model and also prototxt for training. (Also can you explain how can you feed groundtruths to network for training ? )

Dear @junting @kevinmcguinness @agilmor

I have been working this morning mostly on the README.md of our recently accepted paper for CVPR 2016. I would like to make this repo public as soon as possible, so I would need you to tell me if you think that everything contained at the moment is enough to reproduce the experiments.

There are still some formatting issues I can improve, but I think that the basic is there. Also, please check in the source code and check that everything is in there.

Please answer this message with your OK as I will wait for it before launching the repo as well as a github project site based on the README.md you can find here.

What do you think ?

As I am a freshman on deep learning, I have a stupid question about the structure of your shallow net. There are two slices (slice1 and slice2) between the FC layer and the Maxout layer. I'm not clear about what they are. Are they the neurons in the hidden layer of the Maxout layer? I will be very appreciate it if you can answer that.

Hi, I'm now using the deep model with Caffe. However, when I tried to load the net an error always occurs: Check failed: target_blobs[j]->num() == source_layer.blobs(j).num() (96 vs. 0) I suspect that the deploy file and the model are not correspond, are they? The deploy file is on this github project and my code is here:

import caffe
import glob
import os
import numpy as np
from scipy import misc

inputFolder = r'../../image'
outputFolder = r'./results'
modelFile = 'deep_net_model.caffemodel'
deployFile = 'deep_net_deploy.prototxt'

if os.path.isdir(outputFolder) == False:
    os.mkdir(outputFolder)

net = caffe.Net(deployFile, modelFile, caffe.TEST)

It stops at caffe.Net().

I found that the demo code does not work well as is - especially for deep model, so I fixed some minor bugs - especially to_rgb function.

Anyway, the work is amazing and I was impressed! 👍

Hi, We are trying to run your code. But we couldn't find "JuntingNet_iSUN.pickle" input file which is used in demo.py of shallow folder. Can you please help us by providing link to required file. Thank you.

Hey,

I was trying to re-use your deep-net caffe model in my own Torch network to compare saliency maps from two seperate inputs using KL Divergence loss. (Note that I do-not fine-tune your model while training my network; it is simply used for outputting saliency maps and backward propagating the loss through the gradients wrt its input).

I am curious to know if its simply a plug and play model (apart from taking care of input characteristics of a caffe model like BGR format and input value ranging b/w 0 -> 255) or are there some other pre and post processing steps involved in utilising the model.

This is because I see that in the init.py file, you apply various pre-processing as well as post processing to the input image and to the Saliency Maps (as mentioned in your paper), but I am not clear if they are to be applied for test purposes as well ?

Thanks in advance for your response to this thread.

Hi,

I am reading your paper. For the Table 5 of your paper, you report the AUC_Judd and AUC_Borji.

However, when we submit the results to the SALICON official website, they only give the AUC_Borji value of SALICON test set.

Could you please tell me how did you obtain the AUC_Judd value?

Bests.

As a beginner in caffe and deep learning, l am exploring how to train a deep net. So could you share the training code of deep CNN for saliency predction？

Hi, I am trying to implement the DeepNet architecture in pytorch. The code seems to work fine but the result are not as expected. I have done as per the protext files which are provided in the issue 3 and 9. You can find my implementation in https://github.com/Goutam-Kelam/Visual-Saliency/tree/master/Deep_Net. It would be helpful if you can tell me where my mistake lies. Thankyou in advance

Dear Pan,

I am Sen, a phd student from the university of Exeter. Recently I am doing the literature review of saliency prediction and found your work. Therefore, I want to reproduce your work using the code you provided on Github. But when I was training using your code, the training loss is still zero, and the validation loss is also very small. It seems that I was training a pretrained model. So is there any missing file on github for training your shallow network. And another question is the input dimension. What is the dimension of X, and y in train.py of the shallow file? Have you done any preprocessing for the input image and the ground truth? Thank you very much!

Hey, Nowdays, I have read your article about detecting image saliency published on CVPR 2016. Now I have a question: How to custmize new loss function of CNN in Lasagne&Theano specifically? Great thanks in advance for your response.

Dear friends: I have a question about the the experiment results of the shallow convolutional networks for saliency prediction. I use the code of shallow network provided by https://github.com/imatge-upc/saliency-2016-cvpr and use the default parameters, but we cannot get the result provided by the paper (Shallow and Deep Convolutional Networks for Saliency Prediction). Almost all the result generated by the Shallow network are only have the hot spot in the center of image.

Some results is avaliable in the attach files, can you give me some advise about this experiment?

partof the result.zip