@article{Szegedy:2013vw,
author = {Szegedy, C and Zaremba, W and Sutskever, I and Bruna, J and Erhan, D and Goodfellow, I and Fergus, Rob},
title = {{Intriguing properties of neural networks}},
journal = {ArXiv e-prints},
year = {2013},
volume = {cs.CV},
month = dec,
annote = {Good analysis paper, showing that 1) at higher layers, units or mixture of units are roughtly the same. BUT, I'm not sure if this is still the case for networks with only one FC layer (only fc8, in AlexNet's term) 2) adversarial examples. Their root reason might be that each layer is not stable (Table 5).



Section 4

> a non-local generalization prior over the input space. In other words, it is assumed that is possible for the output unit to assign non- significant (and, presumably, non-epsilon) probabilities to regions of the input space that contain no training examples in their vicinity.

this is also mentioned in DL book. DL models assumes some non-local prior.





Section 4.1 here they assume images pixels have range 0 to 1, not 0-255.

Here D(x,f(x)) should be x, not f(x). clearly f(x) doesn't makes dimension match.


Here how they actually find an adversarial example. is really vague. It's explained more clearly in [Exploring the Space of Adversarial Images](https://arxiv.org/pdf/1510.05328.pdf)  (version 5 on ArXiv) with code at <https://github.com/tabacof/adversarial>. Later referred to "the 2015 paper". I copied relevant page in the notes.

Essentially, you need to first select an wrong label l to fool the network, and then you start from a small C (this paper), or big C (the 2015 paper), and then surely you will find a somewhat big r that will satisfy that f(x+r)=l. Then you decrease C little by little, until that f(x+r) != l. This process will intuitively decrease r, yet biasing the r to those preserving f(x+r)=l. Intuitively makes sense. Not sure how good it is in theory.


> This penalty function method would yield the exact solution for D(X,l) in the case of convex losses, however neural networks are non-convex in general, so we end up with an approximation in this case.

I don't think this is right. At least I can't derive it using my RUBBISH note.



======RUBBISH START======

I don't understand the math of box-constrained L-BFGS. I think here it's using the correspondence between Constrained and Lagrange forms. See text around Eq. 5.7 and 5.8 of [Statistical Learning with Sparsity](https://trevorhastie.github.io/): "For convex programs, the Lagrangian allows for the constrained problem (5.5) to be solved by reduction to an equivalent unconstrained problem.", or page 16-17 of <http://www.stat.cmu.edu/{\textasciitilde}ryantibs/convexopt-S15/lectures/12-kkt.pdf>

Here I would say the correspondence is loose. to make it more precise, we should modify the constraint f(x+r)=l in the first (original) problem to be something like loss(x+r,l)<0.001.

Two problems, written as constrained form, and lagrange form, are

1) constrained form.

min |r|, s.t. loss(x+r,l) < \eps

2) lagrange form

min |r| + \lambda loss(x+r,l).

First, we replace hard constraint f(x+r)=l with a more soft constraint using loss (I would say the paper is poorly written; no relationship between f and loss_f is mentioned).

Then they also assume that, big eps correspond to small lambda, and small eps correspond to big lambda.    This assumption is also kind of assumed in Section 7.2 of Deep Learning book. Check my notes on that.

======RUBBISH END======




Section 4.2

> A subtle, but essential detail is that we only got improvements by generating adversarial examples for each layer outputs which were used to train all the layers above. The network was trained in an alternating fashion, maintain- ing and updating a pool of adversarial examples for each layer separately in addition to the original training set.

For adversarial example to be useful, you can't use input-level adversarial examples. Instead, you need intermediate adversarial examples. But this would really make training complicated.

Section 4.3

Not sure about the math about deriving operator norm of W when W is convolution. But whatever... Conclusion is important.},
keywords = {deep learning, To Read},
read = {Yes},
rating = {4},
date-added = {2017-03-29T19:52:30GMT},
date-modified = {2017-03-30T14:48:37GMT},
url = {http://arxiv.org/abs/1312.6199},
local-url = {file://localhost/Users/yimengzh/Documents/Papers3_revised/Library.papers3/Articles/2013/Szegedy/arXiv%202013%20Szegedy.pdf},
file = {{arXiv 2013 Szegedy.pdf:/Users/yimengzh/Documents/Papers3_revised/Library.papers3/Articles/2013/Szegedy/arXiv 2013 Szegedy.pdf:application/pdf}},
uri = {\url{papers3://publication/uuid/C6374003-66D2-4059-941E-D3B1E4F50BBB}}
}

{"_id":"yoLwsjLWX9Qs5j2nQ","authorIDs":["547ccb28a29145d03f000113","593951f1e428c4f601000047"],"author_short":["Szegedy, C","Zaremba, W","Sutskever, I","Bruna, J","Erhan, D","Goodfellow, I","Fergus, R."],"bibbaseid":"szegedy-zaremba-sutskever-bruna-erhan-goodfellow-fergus-intriguingpropertiesofneuralnetworks-2013","bibdata":{"bibtype":"article","type":"article","author":[{"propositions":[],"lastnames":["Szegedy"],"firstnames":["C"],"suffixes":[]},{"propositions":[],"lastnames":["Zaremba"],"firstnames":["W"],"suffixes":[]},{"propositions":[],"lastnames":["Sutskever"],"firstnames":["I"],"suffixes":[]},{"propositions":[],"lastnames":["Bruna"],"firstnames":["J"],"suffixes":[]},{"propositions":[],"lastnames":["Erhan"],"firstnames":["D"],"suffixes":[]},{"propositions":[],"lastnames":["Goodfellow"],"firstnames":["I"],"suffixes":[]},{"propositions":[],"lastnames":["Fergus"],"firstnames":["Rob"],"suffixes":[]}],"title":"Intriguing properties of neural networks","journal":"ArXiv e-prints","year":"2013","volume":"cs.CV","month":"December","annote":"Good analysis paper, showing that 1) at higher layers, units or mixture of units are roughtly the same. BUT, I'm not sure if this is still the case for networks with only one FC layer (only fc8, in AlexNet's term) 2) adversarial examples. Their root reason might be that each layer is not stable (Table 5). Section 4 > a non-local generalization prior over the input space. In other words, it is assumed that is possible for the output unit to assign non- significant (and, presumably, non-epsilon) probabilities to regions of the input space that contain no training examples in their vicinity. this is also mentioned in DL book. DL models assumes some non-local prior. Section 4.1 here they assume images pixels have range 0 to 1, not 0-255. Here D(x,f(x)) should be x, not f(x). clearly f(x) doesn't makes dimension match. Here how they actually find an adversarial example. is really vague. It's explained more clearly in [Exploring the Space of Adversarial Images](https://arxiv.org/pdf/1510.05328.pdf) (version 5 on ArXiv) with code at <https://github.com/tabacof/adversarial>. Later referred to \"the 2015 paper\". I copied relevant page in the notes. Essentially, you need to first select an wrong label l to fool the network, and then you start from a small C (this paper), or big C (the 2015 paper), and then surely you will find a somewhat big r that will satisfy that f(x+r)=l. Then you decrease C little by little, until that f(x+r) != l. This process will intuitively decrease r, yet biasing the r to those preserving f(x+r)=l. Intuitively makes sense. Not sure how good it is in theory. > This penalty function method would yield the exact solution for D(X,l) in the case of convex losses, however neural networks are non-convex in general, so we end up with an approximation in this case. I don't think this is right. At least I can't derive it using my RUBBISH note. ======RUBBISH START====== I don't understand the math of box-constrained L-BFGS. I think here it's using the correspondence between Constrained and Lagrange forms. See text around Eq. 5.7 and 5.8 of [Statistical Learning with Sparsity](https://trevorhastie.github.io/): \"For convex programs, the Lagrangian allows for the constrained problem (5.5) to be solved by reduction to an equivalent unconstrained problem.\", or page 16-17 of <http://www.stat.cmu.edu/~ryantibs/convexopt-S15/lectures/12-kkt.pdf> Here I would say the correspondence is loose. to make it more precise, we should modify the constraint f(x+r)=l in the first (original) problem to be something like loss(x+r,l)<0.001. Two problems, written as constrained form, and lagrange form, are 1) constrained form. min |r|, s.t. loss(x+r,l) < \\eps 2) lagrange form min |r| + λ loss(x+r,l). First, we replace hard constraint f(x+r)=l with a more soft constraint using loss (I would say the paper is poorly written; no relationship between f and loss_f is mentioned). Then they also assume that, big eps correspond to small lambda, and small eps correspond to big lambda. This assumption is also kind of assumed in Section 7.2 of Deep Learning book. Check my notes on that. ======RUBBISH END====== Section 4.2 > A subtle, but essential detail is that we only got improvements by generating adversarial examples for each layer outputs which were used to train all the layers above. The network was trained in an alternating fashion, maintain- ing and updating a pool of adversarial examples for each layer separately in addition to the original training set. For adversarial example to be useful, you can't use input-level adversarial examples. Instead, you need intermediate adversarial examples. But this would really make training complicated. Section 4.3 Not sure about the math about deriving operator norm of W when W is convolution. But whatever... Conclusion is important.","keywords":"deep learning, To Read","read":"Yes","rating":"4","date-added":"2017-03-29T19:52:30GMT","date-modified":"2017-03-30T14:48:37GMT","url":"http://arxiv.org/abs/1312.6199","local-url":"file://localhost/Users/yimengzh/Documents/Papers3_revised/Library.papers3/Articles/2013/Szegedy/arXiv%202013%20Szegedy.pdf","file":"arXiv 2013 Szegedy.pdf:/Users/yimengzh/Documents/Papers3_revised/Library.papers3/Articles/2013/Szegedy/arXiv 2013 Szegedy.pdf:application/pdf","uri":"˘rlpapers3://publication/uuid/C6374003-66D2-4059-941E-D3B1E4F50BBB","bibtex":"@article{Szegedy:2013vw,\nauthor = {Szegedy, C and Zaremba, W and Sutskever, I and Bruna, J and Erhan, D and Goodfellow, I and Fergus, Rob},\ntitle = {{Intriguing properties of neural networks}},\njournal = {ArXiv e-prints},\nyear = {2013},\nvolume = {cs.CV},\nmonth = dec,\nannote = {Good analysis paper, showing that 1) at higher layers, units or mixture of units are roughtly the same. BUT, I'm not sure if this is still the case for networks with only one FC layer (only fc8, in AlexNet's term) 2) adversarial examples. Their root reason might be that each layer is not stable (Table 5).\n\n\n\nSection 4\n\n> a non-local generalization prior over the input space. In other words, it is assumed that is possible for the output unit to assign non- significant (and, presumably, non-epsilon) probabilities to regions of the input space that contain no training examples in their vicinity.\n\nthis is also mentioned in DL book. DL models assumes some non-local prior.\n\n\n\n\n\nSection 4.1 here they assume images pixels have range 0 to 1, not 0-255.\n\nHere D(x,f(x)) should be x, not f(x). clearly f(x) doesn't makes dimension match.\n\n\nHere how they actually find an adversarial example. is really vague. It's explained more clearly in [Exploring the Space of Adversarial Images](https://arxiv.org/pdf/1510.05328.pdf) (version 5 on ArXiv) with code at <https://github.com/tabacof/adversarial>. Later referred to \"the 2015 paper\". I copied relevant page in the notes.\n\nEssentially, you need to first select an wrong label l to fool the network, and then you start from a small C (this paper), or big C (the 2015 paper), and then surely you will find a somewhat big r that will satisfy that f(x+r)=l. Then you decrease C little by little, until that f(x+r) != l. This process will intuitively decrease r, yet biasing the r to those preserving f(x+r)=l. Intuitively makes sense. Not sure how good it is in theory.\n\n\n> This penalty function method would yield the exact solution for D(X,l) in the case of convex losses, however neural networks are non-convex in general, so we end up with an approximation in this case.\n\nI don't think this is right. At least I can't derive it using my RUBBISH note.\n\n\n\n======RUBBISH START======\n\nI don't understand the math of box-constrained L-BFGS. I think here it's using the correspondence between Constrained and Lagrange forms. See text around Eq. 5.7 and 5.8 of [Statistical Learning with Sparsity](https://trevorhastie.github.io/): \"For convex programs, the Lagrangian allows for the constrained problem (5.5) to be solved by reduction to an equivalent unconstrained problem.\", or page 16-17 of <http://www.stat.cmu.edu/{\\textasciitilde}ryantibs/convexopt-S15/lectures/12-kkt.pdf>\n\nHere I would say the correspondence is loose. to make it more precise, we should modify the constraint f(x+r)=l in the first (original) problem to be something like loss(x+r,l)<0.001.\n\nTwo problems, written as constrained form, and lagrange form, are\n\n1) constrained form.\n\nmin |r|, s.t. loss(x+r,l) < \\eps\n\n2) lagrange form\n\nmin |r| + \\lambda loss(x+r,l).\n\nFirst, we replace hard constraint f(x+r)=l with a more soft constraint using loss (I would say the paper is poorly written; no relationship between f and loss_f is mentioned).\n\nThen they also assume that, big eps correspond to small lambda, and small eps correspond to big lambda. This assumption is also kind of assumed in Section 7.2 of Deep Learning book. Check my notes on that.\n\n======RUBBISH END======\n\n\n\n\nSection 4.2\n\n> A subtle, but essential detail is that we only got improvements by generating adversarial examples for each layer outputs which were used to train all the layers above. The network was trained in an alternating fashion, maintain- ing and updating a pool of adversarial examples for each layer separately in addition to the original training set.\n\nFor adversarial example to be useful, you can't use input-level adversarial examples. Instead, you need intermediate adversarial examples. But this would really make training complicated.\n\nSection 4.3\n\nNot sure about the math about deriving operator norm of W when W is convolution. But whatever... Conclusion is important.},\nkeywords = {deep learning, To Read},\nread = {Yes},\nrating = {4},\ndate-added = {2017-03-29T19:52:30GMT},\ndate-modified = {2017-03-30T14:48:37GMT},\nurl = {http://arxiv.org/abs/1312.6199},\nlocal-url = {file://localhost/Users/yimengzh/Documents/Papers3_revised/Library.papers3/Articles/2013/Szegedy/arXiv%202013%20Szegedy.pdf},\nfile = {{arXiv 2013 Szegedy.pdf:/Users/yimengzh/Documents/Papers3_revised/Library.papers3/Articles/2013/Szegedy/arXiv 2013 Szegedy.pdf:application/pdf}},\nuri = {\\url{papers3://publication/uuid/C6374003-66D2-4059-941E-D3B1E4F50BBB}}\n}\n","author_short":["Szegedy, C","Zaremba, W","Sutskever, I","Bruna, J","Erhan, D","Goodfellow, I","Fergus, R."],"key":"Szegedy:2013vw","id":"Szegedy:2013vw","bibbaseid":"szegedy-zaremba-sutskever-bruna-erhan-goodfellow-fergus-intriguingpropertiesofneuralnetworks-2013","role":"author","urls":{"Paper":"http://arxiv.org/abs/1312.6199"},"keyword":["deep learning","To Read"],"metadata":{"authorlinks":{}},"downloads":0,"html":""},"bibtype":"article","biburl":"https://leelabcnbc.github.io/lab-wiki-builder/demo_ref/computer_vision/analysis_visualization/bib.bib","creationDate":"2015-04-08T04:41:23.539Z","downloads":0,"keywords":["deep learning","to read"],"search_terms":["intriguing","properties","neural","networks","szegedy","zaremba","sutskever","bruna","erhan","goodfellow","fergus"],"title":"Intriguing properties of neural networks","year":2013,"dataSources":["WcLjkMsFkC28p8A3T","bzxc3uBcwMv3h47xE","pctN4WRc5riTGuJ4f","KHfEoGnXbz3vBDpc6","pzyFFGWvxG2bs63zP","KZFesWbmGy4yc4ZLC","zk5iRCnLKHPqvtWEk"]}