CS2125 Paper Review Form - Winter 2019

Reviewer: Yilin Han    

Paper Title: Semantic Adversarial Deep Learning

Author(s): Tommaso Dreossi, Somesh Jha, and Sanjit A. Seshia

1) Is the paper technically correct?
 [X] Yes
 [ ] Mostly (minor flaws, but mostly solid)
 [ ] No

2) Originality
 [ ] Very good (very novel, trailblazing work)
 [ ] Good 
 [X] Marginal (very incremental)
 [ ] Poor (little or nothing that is new)

3) Technical Depth
 [ ] Very good (comparable to best conference papers)
 [ ] Good (comparable to typical conference papers)
 [X] Marginal depth
 [ ] Little or no depth

4) Impact/Significance
 [ ] Very significant
 [ ] Significant
 [X] Marginal significance.
 [ ] Little or no significance.

5) Presentation
 [ ] Very well written
 [ ] Generally well written
 [ ] Readable
 [X] Needs considerable work
 [ ] Unacceptably bad

6) Overall Rating
 [ ] Strong accept (award quality)
 [ ] Accept (high quality - would argue for acceptance)
 [ ] Weak Accept (borderline, but lean towards acceptance)
 [X] Weak Reject (not sure why this paper was published)


7) Summary of the paper's main contribution and rationale
   for your recommendation. (1-2 paragraphs)

This paper mainly discussed three parts. It starts with a survey that discussed the main adversarial attacks and defenses that exists for machine learning. In the second part, it proposed a formal method called compositional falsification framework that can be used to verify the security of the machine learning models. The last part, it discussed the experiments on the testing of the machine learning model. 

I do not recommend this paper since it is a conference invited paper. Three authors mentioned their work perspective on machine learning verification. However, the paper itself did not end up with any useful outcome. I would suggest people read this paper "Compositional falsification of ´
cyber-physical systems with machine learning components", because this covers the details of the compositional falsification framework, and at least provides a good way to measures the machine learning model. 

8) List 1-3 strengths of the paper.  (1-2 sentences each,
identified as S1, S2, S3.)

S1: Compositional falsification framework is a promising idea in verification of machine learning models. 

9) List 1-3 weaknesses of the paper (1-2 sentences each,
identified as W1, W2, W3.)

W1: The paper does not discuss any technical details. It should be more comprehensive.