CS2125 Paper Review Form - Winter 2019

Reviewer: Nick Feng

Paper Title: AI2: Safety and Robustness Certification of Neural Networks with Abstract Interpretation  

Author(s):Timon Gehr, Matthew Mirman, Dana Drachsler-Cohen, Petar Tsankov, Swarat Chaudhuri∗, Martin Vechev

1) Is the paper technically correct?
 [x] Yes
 [ ] Mostly (minor flaws, but mostly solid)
 [ ] No

2) Originality
 [x] Very good (very novel, trailblazing work)
 [ ] Good 
 [ ] Marginal (very incremental)
 [ ] Poor (little or nothing that is new)

3) Technical Depth
 [ ] Very good (comparable to best conference papers)
 [x] Good (comparable to typical conference papers)
 [ ] Marginal depth
 [ ] Little or no depth

4) Impact/Significance
 [ ] Very significant
 [x] Significant
 [ ] Marginal significance.
 [ ] Little or no significance.

5) Presentation
 [ ] Very well written
 [x] Generally well written
 [ ] Readable
 [ ] Needs considerable work
 [ ] Unacceptably bad

6) Overall Rating
 [x] Strong accept (award quality)
 [ ] Accept (high quality - would argue for acceptance)
 [ ] Weak Accept (borderline, but lean towards acceptance)
 [ ] Weak Reject (not sure why this paper was published)


7) Summary of the paper's main contribution and rationale
   for your recommendation. (1-2 paragraphs)

   Under the threat of adversarial example attacks, this paper presented a novel approach for verifying 
   Neural Networks' safety and robustness properties by leveraging classic Abstract Interpretation (AI). 
   Because verifying neural network with all possible concrete examples is impractical, AI2 converts the problem
   into abstract domain and uses abstract transformation to obtain a sound over-approximation of neural network's 
   behaviors. The authors presented an abstraction model for a simple Convolution Neural Network (CNN) with 2 abstraction
   transformation functions to approximate 3 concrete CAT transformation layers in the network. The authors also presented a SMT-based 
   method for verifying linear robustness property. The author argues that the choice of the abstract domain 
   affects abstract Interpretation’s precision and scalability and AI2's experiment results on robustness property demonstrated 
   the trade-off between precision and scalability. The benchmarks performance comparsion agasint Reluplex on FNN
   indicated that AI2 is more effective on large FNN networks, and AI2 is also the first tool supporting verification on CNN.





   




8) List 1-3 strengths of the paper.  (1-2 sentences each,
identified as S1, S2, S3.)
S1. A clear and comprehensive presentation of background knowledge on Convolution Neural Network and Abstract Interpretation.
S2. A strong argument on the rationale for using abstract Interpretation for verifying robustness properties. 
S3. By expressing CNN as conditional affine transformation (CAT), the paper presented a convincing argument 
that abstract affine functions and abstract case function are sound abstract transformation functions for neural networks. 



9) List 1-3 weaknesses of the paper (1-2 sentences each,
identified as W1, W2, W3.)

W1. The paper focus on proving robustness property for neural networks with known adversarial attack methods. However, in practice, the adversarial 
attack method as well as the defending robustness properties are unknown.  
W2. Abstract transformation introduces over-approximation between each step that will eventually affect the overall precision of the method. However,
the paper failed to deliver a strong argument on the degree or bound of over-approximation for their proposed transformation functions.  An adversarial example
could target the transformation functions' imprecision to create unverifiable properties. 
W3. The paper assumes Convolution Neural Network(CNN) can be expressed as conditional affine transformation (CAT). Even though 
this is true for simple CNN with RELU activation, CAT is not strong enough to express general activation function (logistic, TanH). 
Therefore, the applicability of AI2 is challenged when the verifying the network has general non-linear elements.