From: Steve Easterbrook Date: Mon, Feb 3, 2014 at 6:09 PM Subject: straw man proposal for grad depth oral requirements To: faculty@cs.toronto.edu Folk, In last Tuesday's faculty meeting, we discussed the issue with long completion times among our PhD students, and how the current set of thesis committee milestones don't appear to be helping students to pass through the program at a reasonable pace. We talked about possible changes to the overall structure of thesis committee milestones, but didn't achieve a consensus on these. I would like to propose one simple change that I think will help to mitigate the problem, and that we could implement immediately, while we work out a longer term solutions. As I mentioned in the meeting, many students I've talked to are confused about the requirements for the qualifying ("depth") oral, and get very different messages about it from different faculty members. Some of us tell students to write a detailed literature survey, expecting it to be the basis for chapter 2 of the thesis. Some expect a synthesis of related work that could be a publishable contribution in its own right. Some ask for a detailed analysis of dozens of key papers related to the problem the student has chosen for her thesis, with an emphasis on characterizing the open problems. I think most of this advice is unhelpful, as it causes students to delay scheduling this milestone until far too late. In particular, it requires a student to have picked a thesis topic, and to have completed most of the work we expect at the next milestone, the research proposal. I propose that we make a clearer distinction between the depth oral and the research proposal milestones, by making it explicit that the student does *not* have to have selected a thesis topic to pass the qualifying oral. I also think we should be much clearer about what we *are* looking for at this stage - i.e. specific research skills in evaluating the literature. Here's a strawman proposal for re-writing the guidance for the qualifying oral: ------- To prepare for the qualifying "depth" oral, the student should produce a written report that analyzes, in detail, the research of a handful (i.e. 4 to 6) of papers in a selected research area. The selected area need not correspond to the student's eventual choice of PhD topic, and hence the student does not have to have picked a thesis topic prior to the qualifying oral. The paper should demonstrate the student has mastered the following research skills: a) The ability to select a small number of significant research contributions from a larger set of published papers, and justify that selection. b) The ability to articulate a rationale for selection of these papers, on the basis of significance of the results, novelty of the approach, etc. c) The ability to relate the papers to one another, and to other research in the literature. d) The ability to critique the research methods used in these papers, the strengths and weaknesses of these methods, and likely threats to validity, whether acknowledged in the papers or not. e) The ability to suggest alternative approaches to answering the research questions posed in these papers. f) The ability to identify limitations on the results reported in the papers, along with their implications. g ) The ability to identify and prioritize lines of investigation for further research, based on limitations of the research described in the papers and/or important open problems that the papers fail to answer. Note that to complete this milestone, the student will need to have read many more than the 4-6 papers described in the depth oral report, and be prepared to discuss some of these other papers (in broad terms), and the reasons for *not* including them, in response to questions from the committee. The examination typically lasts approximately two hours and begins with a 20 to 30 minute presentation by the candidate. The committee will determine whether the student should proceed to thesis work. They may recommend that the student do more course work or reading. ------- Steve PS For reference, here's the current description in the grad handbook: "The student presents an area of research to the supervisory committee, normally in a closed forum. The purpose of this examination is to assess the student's understanding of the literature in the area of research, as well as preparedness to do research in that area. This involves assimilating the significant research papers on the topic, understanding how they relate to one another, and identifying valid open research questions. The student typically prepares a short written survey of the work in the area, and distributes it to the supervisory committee at least two weeks prior to the examination. The length of the survey should not exceed that appropriate for inclusion in a doctoral thesis in the area. Material written by the candidate for another purpose (for example, the research paper) may be re-used in the survey. The area chosen by the student should be sufficiently broad to contain many potential thesis topics, yet sufficiently narrow that the highly relevant papers number in the tens rather than in the hundreds. The examination typically lasts approximately two hours and begins with a 20 to 30 minute presentation by the candidate. The committee will determine whether the student should proceed to thesis work. They may recommend that the student do more course work or reading."