CS2125 Paper Review Form - Winter 2018 Reviewer: Laura Walsh Paper Title: Why looking isn't always seeing: readership skills and graphical programming Author(s): M. Petre 1) Is the paper technically correct? [x] Yes [ ] Mostly (minor flaws, but mostly solid) [ ] No 2) Originality [ ] Very good (very novel, trailblazing work) [x] Good [ ] Marginal (very incremental) [ ] Poor (little or nothing that is new) 3) Technical Depth [ ] Very good (comparable to best conference papers) [x] Good (comparable to typical conference papers) [ ] Marginal depth [ ] Little or no depth 4) Impact/Significance [ ] Very significant [x] Significant [ ] Marginal significance. [ ] Little or no significance. 5) Presentation [x] Very well written [ ] Generally well written [ ] Readable [ ] Needs considerable work [ ] Unacceptably bad 6) Overall Rating [ ] Strong accept (award quality) [x] Accept (high quality - would argue for acceptance) [ ] Weak Accept (borderline, but lean towards acceptance) [ ] Weak Reject (not sure why this paper was published) 7) Summary of the paper's main contribution and rationale for your recommendation. (1-2 paragraphs) This paper highlights the fact that that even though a graphical representation of a program may be appealing, it is most likely less effective in conveying its meaning than a textual representation. The author of this paper ran a study where she gave experts and novices a set of programs, some expressed diagrammatically and some textually. The results revealed that graphical representations are more difficult to understand, among both expert and novice users, than their textual equivalents. The concepts were presented and laid out well. The paper is very well written and draws the reader in with a logical progression in the flow of ideas. Overall, the author makes the point that textual and graphical representations both have their strong points and weaknesses. She suggests that a graphical representation can be made better by adding secondary notation which gives the user cues about how to interpret it. However, an important point is also made that the addition of secondary notation has the potential to make interpreting a diagram even more confusing should it give misleading cues. The author explores many factors that can play a role in how graphical representations are used, such as the amount of experience a user has or the type of training they have received. The main takeaway is that diagrammatic and textual representations can both be effective tools of communication: the important consideration is to make sure the type of representation is best suited for the specific application in question. 8) List 1-3 strengths of the paper. (1-2 sentences each, identified as S1, S2, S3.) S1. The simple program examples that are interspersed throughout the paper are helpful in bringing across the author's point that different representations of the same logic are processed and understood in different ways. S2. The author ran a study whose results were corroborated by other referenced experiments, adding confidence in the accuracy of the findings. S3. The author took into account what the experts who participated in the study liked about graphical representations and commented diplomatically on each of their points. It was interesting to read about how people's perception that a graphical representation is helpful can make it more appealing, even if the results show that it is still easier to interpret a textual representation. 9) List 1-3 weaknesses of the paper (1-2 sentences each, identified as W1, W2, W3.) W1. The study that the author ran on the experts and novices to see how well they could understand what a program was doing was not described in much depth. It would have been interesting to read a more detailed account of that study. W2. The study only compared the efficiency with which the participants were able to understand the program. It would be illuminating to see the results of comparing graphical and textual representations in other areas than just ease of understanding.