CS2125 Paper Review Form - Winter 2019

Reviewer: Hazem Ibrahim

Paper Title: Autonomous Vehicle Safety: An Interdisciplinary Challenge

Author(s): Philip Koopman, Michael Wagner

1) Is the paper technically correct?
 [X] Yes
 [ ] Mostly (minor flaws, but mostly solid)
 [ ] No

2) Originality
 [ ] Very good (very novel, trailblazing work)
 [ ] Good 
 [ ] Marginal (very incremental)
 [X] Poor (little or nothing that is new) (but this is because it is more of a background paper)

3) Technical Depth
 [ ] Very good (comparable to best conference papers)
 [ ] Good (comparable to typical conference papers)
 [X] Marginal depth
 [ ] Little or no depth

4) Impact/Significance
 [ ] Very significant
 [ ] Significant
 [X] Marginal significance.
 [ ] Little or no significance.

5) Presentation
 [X] Very well written
 [ ] Generally well written
 [ ] Readable
 [ ] Needs considerable work
 [ ] Unacceptably bad

6) Overall Rating
 [ ] Strong accept (award quality)
 [X] Accept (high quality - would argue for acceptance)
 [ ] Weak Accept (borderline, but lean towards acceptance)
 [ ] Weak Reject (not sure why this paper was published)


7) Summary of the paper's main contribution and rationale
   for your recommendation. (1-2 paragraphs)

This paper, written by Philip Koopman and Michael Wagner from Carnegie Mellon University, aims to highlight the levels of the functional hierarchy of a autonomous vehicle system, as well as some of the current unsolved problems which exist at each level. Starting with safety engineering, the authors note the difficulty of defining a safety certification (such as ISO26262) for self-adaptive inductive learning systems. On the topic of ultra-dependability, they note that in order to ensure such dependability, one would need to improve system robustness for messy environmental situations, which may seem impossible especially at the early stages of deployment. From a software standpoint, the authors note that in order to ensure that the validation set if comprehensive enough to ensure no gaps in system behaviour, they must cover all conceivable operating scenarios, which may not be possible currently. The authors continue this discussion with the areas of computing hardware, testing, security, Human-Computer Interaction, Legality, and finally Social Acceptance.

While several other papers have tackled one or two of these levels of hierarchy, this paper presents the various critical challenges that exist in deploying autonomous driving systems at scale in clear and simple terms. This makes the paper an interesting and important read for individuals who may interested in diving deeper into this area of research, and hence I would recommend it for acceptance. 


8) List 1-3 strengths of the paper.  (1-2 sentences each,
identified as S1, S2, S3.)

S1. Paper gave a fairly decent overview of the various areas one must consider when deploying ADS systems at scale.

S2. The paper was very well written, which allows those unfamiliar with the topic of ADS to quickly get an overview of the big picture surrounding it's deployment. 


9) List 1-3 weaknesses of the paper (1-2 sentences each,
identified as W1, W2, W3.)

W1. Some of the sections, particularly the section on security, could have been expanded on more.