CSC321 Neural Networks and Machine Learning (UTM)

Winter 2020

Home
Piazza
MarkUs
Homework
Projects
Team
Midterm
Exam FAQ

Exam

The exam will be available remotely on April 16th, and is scheduled for 5pm-7pm. The exam will be open book, meaning that you will be able to use the materials on the course website and your notes to complete the questions.

The exam is cumulative, and covers the material from weeks 1-12, including the tutorials, homework, and projects. One way to start studying is to go through the course materials and make a list of topics. Topics that are in the course notes that weren't covered in lecture will not be on the exam.

Exam Logistics

  • The exam paper will be available on April 16th. You will be able to start the exam as early as 4:00pm and as late as 5:45pm without needing any special arrangements. The offset is (a) in case you have another exam on the same day or the following day, and (b) in case of any issues that might delay your start time by a little bit. I recommend starting the exam between 4pm-5:30pm.
  • If you have a reason to want to start your exam earlier or later, please let us know. We're aware that some of you are in different time zones, and you can certainly write the test another time. However, we need to know in advance. Please clear (verify) your exam time with Lisa ASAP and by April 12th.
  • The exam paper will be available on Markus, similar to how your midterm was returned to you. You will receive a URL, and instructions stating that clicking on that URL is equivalent to opening the front cover of your exam. Do not click on the URL by accident, since we are logging the time that you open your exam.
  • We will have different versions of the exam.
  • Once you have opened your exam, save your exam script on your computer. We expect you to disconnect your internet at this point.
  • You'll have two hours to write the exam. Note the time you started the exam, and keep a timer. You can write your exam on paper and scan your solution afterwards. You can type your answers. You can use a combination of techniques, so long as you make it clear to the TA which file(s) to grade
  • If you are typing your solutions, make sure to save your work frequently. We recommend writing your exam on paper unless you are willing to be careful.
  • After you complete the exam, you'll need to upload your solutions to Markus. Markus will accept text files, PDF files, image files. (Update April 14th) Text and PDF files are preferred, since they are much easier to grade. Please do not upload zip files, since that will make your exam impossible to grade on Markus. You can upload multiple files to Markuss (Update April 14th) but mark in your filename which questions can be found in which file.
  • (Update April 14) If you are submitting an image file, check the orientation of the file you are submitting. We will deduct 25% of the grade from any image solutions that are side ways or upside down, since it greatly impedes our ability to grade the work.
  • In addition to your exam files, you'll be asked to sign (or type) a declaration stating the time that you started and completed the exam, that you did not use any unauthorized aid, and that you did not provide or receive any help from another person.
  • You'll have 30 minutes (additional to the 2 hours) to upload your solutions. During this time, you can scan your written files (or take pictures), save and rename your files, connect to the internet and upload your solutions on Markus, and type/sign your declaration and upload it to Markus. You shouldn't change the content of your solutions in any way during this time.

Mock Exam (Updated April 8th)

To get you used to the exam logistics, we'll be holding a mock exam using the same technology between April 9th-April 11th. The test should take 55 minutes to complete (not the original 30 minutes!). You should:

  • Set up as you would during the exam: prepare your notes, course slides, and have paper and pen/pencils available. Have a timer or a clock visible. Have some water handy.
  • Go to Markus. There will be a "mockexam_download" assignment where you can download the link to your mock exam paper. There will also be a "mockexam" assignment for you to submit solutions:
  • Click on "mockexam_download". Then on the "submissions" tab
  • Download the "csc321_mock_link.pdf" file. This file contains instructions and a link to your personalized mock exam. Open the file and read the instructions, but don't click on the link in the file yet until you're ready.
  • Don't share your link with anyone, since access to your exam file will be logged. During the actual exam, there will be different versions of the exam with different questions. Make sure that you complete the questions in your own exam.
  • When you are ready for your timer to start, click on the link to your exam questions. This action is equivalent to opening the front cover of your exam, and starts your 55 minutes.
  • Save the exam questions locally, disconnect from the internet as you would during the actual exam.
  • Write the exam, spending up to 55 minutes. As a sanity check, we'll check that your final submission time is no more than (55 + 30) minutes.
  • Upload your solutions to the Markus assignment "mockexam", including a mock declaration. I recommend timing this part to make sure you're not exceeding 30 minutes. After this exercise, if you are worried that 30 minute time might not be enough, please email Lisa to discuss your situation.

We won't be grading the mock exam, and we won't be posting solutions. You're welcome to discuss the solutions on Piazza and during office hours, but bear in mind that some students might not have completed the mock exam yet.

We'll do some sanity checks (e.g. for timing, and others checks TBD). If you would like to be included for any sanity checks we do, please submit your exam solutions and declaration by April 11th (midnight).

FAQ

Q: Can I write the test in my time zone?

A: Yes! Please email Lisa so we know when you are intending to write the test.

Q: What happens if my internet cuts off?

A: You shouldn't be using internet during the exam, so that shouldn't be a problem. If you can have a backup plan that you can reasonably execute within the allotted 30 minutes, that would also solve your problem. If you have really spotty internet and anticipate issues, please let Lisa know in advance, and we'll follow up with instructions

Q: Is this test going to be harder because we made it open-book?

A: We're aiming for the same difficulty as the past exams, past midterms, and our midterm. The fact that the test is open-book doesn't change the plan. The fact that our test is open-book means that you won't see many definition questions, and more questions asking you to solve problems. You'll still need to know the meaning of the terminology we used throughout the term to understand the problems. So, the fact that this test is open book doesn't change the amount of materials you need to know.

Q: What if we have a question about the test question?

A: We won't be answering questions. If you think that there are multiple, legitimate ways to interpret an exam question, the best thing to do is write down your assumptions. When grading, we'll take a look at whether your interpretation makes sense given the course materials.

Q: What if I don't feel well on the date of the exam?

A: Let Lisa know, and we'll follow up with next steps. It is very import to not open your exam, i.e. don't click on the exam link. Opening the exam link is equivalent to flipping open the exam cover page at an in-person exam.

Q: Will different versions of the test be significantly different in terms of difficulty?

A: We are using a machine learning model to predict how challenging each exam question will be. The issue is no different from when we have different versions of the midterm, and we did fairly well in making balanced midterm versions. We'll also have questions that we know will be similar in terms of difficulty (e.g. same type of problem but different numbers) to verify our assumptions, and to make adjustments if necessary.

Q: How do you study for an open book test?

A: Pretty much as you would any other exam. There are some helpful resources on test and exam preparation here particularly the PDF link on "Exam Tips" and "Take-home and Online Exams". Keep in mind that you won't have time to look up every single topic, and you certainly won't have time to learn something new. If there is a topic where you find the terminology especially confusing, prepare notes/indices on that topic so you have them ready.

Q: Why can't we use internet during the test?

A: We want you to be able to use the course materials and your notes, but not (for example) search for answers on Google. The internet is a clear place to draw the line, and will not disadvantage students with poor internet access.

Q: Will there be coding questions on the test?

A: You might be asked to read code, and understand the architecture or ideas presented in that code. For example, we might ask you to check if some PyTorch code contains errors in the way the model architecture is set up. We don't expect you to remember the PyTorch API, and we don't expect you to catch Python syntax error. We do, for example, expect you to recognize if the number of output channels in a convolution doesn't match the input channels of the next convolutions

Q: Can I upload more than one file as part of my exam solutions?

A: Yes, but please choose good file names, and help your TA find your solutions. Do not submit a zip file, since they are impossible to grade on Markus.

Q: I have an accessibility letter and require an accomodation.

A: In most (all?) cases, accomodations should be straightforward. If you need a computer, you can use one to type your answers. You should also report the actual start and end time of your exam, and note in your declaration that you have an accessibility letter. If you require other accomodation (e.g. if you cannot upload your exam solutions in 30 minutes), please email Lisa ahead of time and we'll work something out.

Q: Another course is running the exam differently. Why can't we do it their way? (OR why can't they do it our way?)

A: Every course is different, and every instructor has tools that they are comfortable with. For this course, some questions are heavily math based so we want you to have the option of writing your solutions, so we didn't go with a Quercus quiz. We're also not comfortable with introducing a new technology that is both new to the class, and that the instructors don't have experience with, so we didn't go with CrowdMark. Some courses have less flexibility than others (e.g. second-year courses, courses that are pre-requisites of other courses...).

Q (new 4/2): What is the reason for having it up for 24 hours if we are to start within 4 - 5:45 pm on April 16th?

A: The reason is that you can start earlier or end later than 4pm/5:45pm if you need to. This is what the "special arrangements" mean. You just need to let Lisa know and get approval, which you'll could get for many reasons (e.g. different time zone, another exam, issues accessing space, event that you can't reschedule, other reasons that we might not have thought of). We just want to strike a good balance between flexibility and preventing academic offences. Knowing who will start earlier/later ahead of time helps us randomize questions, so that people who start later get different versions of the test questions from people who start earlier.

Q (new 4/2): Which resources are we allowed to use for the online final exam?

A: You can use any resources on the course website, including course slides, course notes (by Prof. Roger Grosse and by Lisa), tutorial notes, past assignments and solutions, quercus announcements. You can also use any of notes or resources that you have personally created. Keep in mind that you are not permitted to access the internet while writing the exam, so download the electronic resources that you want to use ahead of time. (We'll have a few way to police this) You won't have time to look up every answer, and we'll be asking you questions that you've never seen before.

Q (new 4/2): How will you know if we are using the internet?

A: We'll be able to check access to the course website, Markus records, hits to the exam URL, etc. You're also required to sign a declaration stating that you have not used unauthorized aids. We're trust in the integerity of the majority of the students. Hopefully, knowing that we'll be checking what we can will be enough to discourage academic offenses.