CS2125 Paper Review Form - Winter 2019

Reviewer: Yilin Han    

Paper Title:Towards Verified Artificial Intelligence

Author(s):Sanjit A. Seshia, Dorsa Sadigh†, and S. Shankar Sastry

1) Is the paper technically correct?
 [X] Yes
 [ ] Mostly (minor flaws, but mostly solid)
 [ ] No

2) Originality
 [ ] Very good (very novel, trailblazing work)
 [X] Good 
 [ ] Marginal (very incremental)
 [ ] Poor (little or nothing that is new)

3) Technical Depth
 [ ] Very good (comparable to best conference papers)
 [ ] Good (comparable to typical conference papers)
 [X] Marginal depth
 [ ] Little or no depth

4) Impact/Significance
 [ ] Very significant
 [ ] Significant
 [X] Marginal significance.
 [ ] Little or no significance.

5) Presentation
 [ ] Very well written
 [ ] Generally well written
 [X] Readable
 [ ] Needs considerable work
 [ ] Unacceptably bad

6) Overall Rating
 [ ] Strong accept (award quality)
 [X] Accept (high quality - would argue for acceptance)
 [ ] Weak Accept (borderline, but lean towards acceptance)
 [ ] Weak Reject (not sure why this paper was published)


7) Summary of the paper's main contribution and rationale
   for your recommendation. (1-2 paragraphs)

This paper focuses on the formal methods perspective that identified five challenges respects to verified of AI: environment modeling, formal specification, system modeling, computational engines, and correct-by-construction design. However, I think except for "computational engines", none of them can be properly addressed from the current ML development. In terms of the principles they proposed to address the challenges, I think if they could address the current drawbacks of the ML, they could make AI verifiable without proposing these principles. 

I will recommend this paper to who is trying to build the theory of verification on machine learning, especially people focus on formal languages. 

8) List 1-3 strengths of the paper.  (1-2 sentences each,
identified as S1, S2, S3.)

S1: This paper focuses on the challenges of verified AI from formal methods perspective. The five main challenges for verified AI. These actually true to apply to all AI products. They even come up with five principles that could potentially guide the verification of AI. 


9) List 1-3 weaknesses of the paper (1-2 sentences each,
identified as W1, W2, W3.)

W1: there is no solution for any of these challenges from the current stage of the development of machine learning and AI. It is the common challenges what well known to all ML participants. These principles are easy to say but extremely hard to implement.