CS2125 Paper Review Form - Winter 2019

Reviewer: Zi Yi Chen

Paper Title: Towards Verified Artificial Intelligence

Author(s): Sanjit A. Seshia, Dorsa Sadigh, and S. Shankar Sastry


1) Is the paper technically correct?
 [ ] Yes
 [X] Mostly (minor flaws, but mostly solid)
 [ ] No

2) Originality
 [ ] Very good (very novel, trailblazing work)
 [X] Good 
 [ ] Marginal (very incremental)
 [ ] Poor (little or nothing that is new)

3) Technical Depth
 [ ] Very good (comparable to best conference papers)
 [ ] Good (comparable to typical conference papers)
 [ ] Marginal depth
 [X] Little or no depth

4) Impact/Significance
 [ ] Very significant
 [ ] Significant
 [X] Marginal significance.
 [ ] Little or no significance.

5) Presentation
 [ ] Very well written
 [X] Generally well written
 [ ] Readable
 [ ] Needs considerable work
 [ ] Unacceptably bad

6) Overall Rating
 [ ] Strong accept (award quality)
 [ ] Accept (high quality - would argue for acceptance)
 [X] Weak Accept (borderline, but lean towards acceptance)
 [ ] Weak Reject (not sure why this paper was published)


7) Summary of the paper's main contribution and rationale
   for your recommendation. (1-2 paragraphs)

In this paper, the authors first gave us a brief introduction on what is AI and why do we want to verify AI system.  They described five different challenges for achieving verified AI: environment modelling, formal specification, modeling systems that learn, computational engines for training, testing, and verification, and correct-by-construction intelligent systems.  And they proposed principles to address those challenges from a formal methods perspective.

This paper is generally well organized, easy to read.  But it contains little technical depth.


8) List 1-3 strengths of the paper.  (1-2 sentences each,
identified as S1, S2, S3.)

S1: This paper introduced us to the challenges of verified AI and ways to tackle those challenges
S2: Authors of this paper used autonomous vehicles as their example application which relates to the course material
S3: Paper is well organized, from background knowledge, to the problem, to the solution and conclusion

9) List 1-3 weaknesses of the paper (1-2 sentences each,
identified as W1, W2, W3.)

W1: This paper only discussed the high level knowledge of verified AI, little technical depth
W2: Authors only gave examples on what applications can the formal method principles be used to tackle the challenges, didn't discuss how they are applied, how well they work and what are the limitations