Subject: The manychessmoves question on Monday's exam From: "Christopher McDonald" I've received emails and heard many comments (on the full spectrum from 'challenging' to 'impossible') that the manychessmoves question was a life-affirming moment for a number of students, and I'm interested to find out why. ?? For your interest (which, I appreciate will have wained) there's a solution to the problem below, with the comments for the markers as to where they can look for marks in your solutions. The question was worth 10 marks, but there's 16 marks identified in this solution. Of course there'll be other ways to solve it, too, but the general approach is to just keep, say, 4 copies of the same programming running until any of them fails, or until we've completed the required total number. Other than the system calls required, and given in the question - fork(), execs(), and wait(), all called in the standard way - most of the solution required some standard header files, checking function arguments, and loops controlled by integer variables. Yes, a problem that you hadn't seen before (and won't be in the Deferred exam), but I don't believe anything as extreme as the comments about it. ----- To put in my $0.02 I would say that the question probably included a lot of information and terms that many students hadn't seen before. The notion that a process can be parallelised (something that is addressed fully in later units), though tangentially mentioned when discussing CPU scheduling may have coloured people's understanding of what to do. Similarly, the idea of running heuristic algorithms that produce different outputs each time might be a little spooky for people that haven't done AAAI. That said I think it was a decent question and will probably prove to be a good mark differentiator between distinction and high distinction students. ------ The premise that if we have the basic tools to undertake a problem, it should be easily doable, is flawed. High school students are taught the basics of complex numbers and calculus. Expecting them to be able to do something like complex analysis, is unrealistic. Although functions such as fork(), execs(), and wait() have been shown in a few lecture slides, using them in a completely different context, is difficult. Questions similar to this, have only appeared in past exams (without solutions). There has been no practical applications of these concepts, other then writing a solution to an exam question, and hoping it is correct. The lack of exposure to, and application of these concepts, paired with using them in a completely new context, made for a very challenging question. Not everyone is good at abstracting concepts to a new context, especially with the added timed pressure of an exam. The question was definitely not impossible, but to people without familiarity to this new context. It might as well have been impossible ------ It is however a necessary skill for computer scientists. In many situations as a programmer you are not going to possess a complete understanding of the domain in which you are working and so an ability to abstract away unnecessary detail is essential. Probably not on the unit outline but a useful thing for people to be made aware of. ------ From: "Christopher McDonald" > To put in my $0.02 I would say that the question probably included a lot > .... Thanks for your reply. Talking with a couple of students yesterday, they stated that their discomfort arose from a question with a 'wall of text' to read. This may be something distinguishing STEM exam questions, from those in the Humanities (often read an esaay, then write an essay), but something students are likely to see more of in 3rd year. I'm unsure if the word or idea of a heuristic is a difficult concept, even if not-yet-studied by students half way through a computing degree (but glad that I changed it from 'non- deterministic algorithm'). I'm confident that many students see heuristic approaches 'every day' when playing a game against a computer opponent, but may not have thought about it much. ----- From: "Christopher McDonald" Thanks for your reply. I don't agree with your analogy at all. Firstly, we're a long way from high school, and this question was not tackling anything as distant (to high school students) as calculus. It was certainly an unseen question (and, yes, under exam time pressure), but it 'only' required an application of concepts seen in the unit, and some problem solving. I'm not seeing what you're describing as a 'new context', unless it's that a parent process can simultaneously have multiple child processes running (just like your shell, and compiler, do) I'd met with about 30 students the week before the exam who, until that time, were employing the study of (many, rote learning) past exam papers as their sole means of study, often asking if a particular question was likely to appear again. Yes, they'd refer to lecture material to find where 'the answer was covered', hoping it was on just a few contiguous pages, and not across multiple lectures. None had ever cut-and-paste the code presented in a lecture (in the grey boxes), turned it into a program, compiled and run it. Similarly, a number of students had emailed me (using a computer) their attempts at past exam questions asking "if it would work". Using a computer. Somehow we need to get across a stronger message about exam preparation (and project submission, another story). I'm unsure of the origins of the wide-held belief that exams should only examine previously seen ideas (previous technical concepts, yes, but not the application of those concepts in different areas). It may be a survival technique as fewer full-time students study full-time. ----- > High school students are taught the basics of complex numbers and > calculus. Expecting them to be able to do something like complex analysis, is unrealistic. I wholeheartedly disagree with this analogy as well. You're using a similar example to draw parallels but taking it way too much to the extreme. Complex analysis is a field which requires it's own complete university unit to learn, with some people even devoting years of research into the topic - it is clearly not reasonable to expect a high school student to do it. This question, however, was well within the realms of what we have covered in the unit, requiring only knowledge of fork(), exec(), wait(), and some problem solving. I also agree with [xx] on this. While it was definitely a challenging question, it's quite normal for exams to have one or two 'differentiator' questions to distinguish the top students from the good students. I personally had some trouble figuring out the proper logic to implement the program, but I did my best given the time constraint, using what I'd learnt throughout the semester (and could remember during the exam). Looking at the marking of the question too, it's quite generous; it seems it's entirely possible for someone to gain 4-5~ marks without even attempting the "multi-core forking" by doing the basics (header files, parameter checking, coherent variables, forking and checking for failed fork, main() ). Overall I think it was quite a good, interesting question, testing if we really understood fork() and its usage by presenting it to us in a different context. There have certainly been questions of unfair difficulty, but in my opinion this was not one of them. ----- A view from an average level student: Despite the question is hard indeed, I really appreciate what I learned in this unit. This unit is definitely the hardest one I currently enrolled but it reaches the standard of what I expected a world top 100 university unit should be. I have learn the c programming in a university in my country and at the end of the semester all I learned is some basic application of some basic c statement. After finished this unit, I managed to finish the projects which could be impossible for me one year ago. I know how to manipulate the pointers-- used to be the most fearing and unknown things for me, and have a basic understanding of how the operating system works. For students, maybe the things they most care is whether they can pass the unit or not. Indeed for me it is important too, but more important thing is I learn how to solve a problem using programming and the joy of solving and learning new things after I spent many hours. It maybe not fair putting a such hard level question for students who didn't expect the exam should be that hard. But it is more like a real world, we can not blame the hardness of the question when we are working. Only thing to do is to keep improving the skill until to be good enough for solving the problems. That is how Darwin addressed in his evolution theory--"Only the ones strong enough could be survived". ----- Just wanted to say that the marking key may seem "generous" to you, but keep in mind this unit is attempted by people who aren't necessarily amazing at programming. 4-5 marks for basic things like error checking and initializing in my opinion is completely valid. Keep in mind that getting 4-5 is the bare minimum to pass. Personally I didn't really understand the question or how to go about solving it and to know that I can still get enough marks to at least pass is good to know. Units aren't meant to be passable only by the most proficient. But being proficient enough to get the other 5 marks is where the challenge (and higher marks) is.