Counting on Frank: Postmortem of an Edutainment Product Maria Klawe Douglas Super Marvin WestromDept. of Computer Science Dept. of Computer Science Dept. of Curriculum Studies Univ. of British Columbia Univ. of British Columbia Univ. of British ColumbiaVancouver, BC V6T 1Z4, Can. Vancouver, BC V6T 1Z4, Can. Vancouver, BC V6T 1Z4, Can. ABSTRACT This paper presents preliminary findings on the design and use of Counting on Frank, a CD-ROM math adventure forages 8-12 published by EA Kids in late 1994. Obtained primarily from log-file data on home and school use, andfrom classroom observations, our results provide insight on some common issues in edutainment design and use. Keywords Edutainment, interface design, computers in education, motivation, games, mathematics, children INTRODUCTION Counting on Frank (CoF) is the first commercial product resulting from the Electronic Games for Education in Mathand Science collaboration between researchers in computer science and education, teachers, and commercial gamedevelopers. In CoF the player's goal is to help a boy, Henry, and his dog Frank, win a contest to guess thenumber of jellybeans in a jar (different each time the game is played) by collecting clues as the reward for solvingmathematical word problems. The main educational activities in CoF are solving the word problems, using theclues to determine the number of jellybeans, and playing four other mathematical strategy games (mathgames), eachwith several variants. The primary entertainment elements are the "click-ons" in each of the eight main scenes inCoF, i.e. objects that when clicked on, trigger a humorous animation that is generally unrelated to the educationalactivities. Other non-educational entertainment elements include animation sequences as rewards for correctanswers to word-problems, humorous dialogue by the main characters, etc. Of course, many children found playing themathgames and other educational activities entertaining as well. As is common in edutainment products, throughoutthe development of CoF conflict and tension arose in attempting to simultaneously meet the product'sentertainment and educational goals. This paper reports on the merit of some of the design choices that were made, aswell as the general effectiveness of CoF. These topics are discussed [1,2,3] in more detail. METHODOLOGY The CoF study took place in three components: long-term classroom integration, controlled school sessions, andhome use. In the long-term classroom integration component, two classrooms of students aged 8-10 (grades3 and 4) were each equipped with four computers. Researchers made weekly visits during the 94/95 schoolyear to observe the students using CoF and a small number of other educational computer games. In these classrooms,CoF was integrated with other mathematics learning through non-computer activities based on CoF. Thecontrolled school sessions involved four classes containing approximately 110 children in grades 4 and 5. In eachclass, half of the class played CoF in teams of 3 on computers elsewhere in the school for two 40 minutesessions per week for four weeks, while the other half remained in the classroom doing paper and pencilworksheets of word problems similar to those in CoF. The groups were then switched and the sessions were repeatedfor another four weeks. The participants also completed questionnaires and word-problem tests at four points in thestudy (beginning, middle, end, and six weeks later) to assess changes in attitude and performance. In the homeuse component, nine children aged 9-11 took home a computer with CoF for 2 weeks during their school'sspring break. In all components, none of the children had played CoF before participating in the study. The childrentaking part in the second two components of the study used a modified version of CoF in which all their actions wererecorded in log files, allowing us to conveniently collect and analyze much more detailed information on theplaying of CoF than would have been possible through field observations alone. The modified version of CoF wasnot available until spring of 95 and hence log files were not available for the classroom integration component. Asfar as we know, this study is the first instance where log files have been available for a commercial edutainmentproduct. At the end of the controlled school component we also conducted sociogram surveys focusing on various factors and used observational information to rate teamcohesiveness. PATTERNS OF PLAY A major concern during the design of CoF was whether players would continue to spend significant time on themathematical activities, or would just play with click-ons. Our study found this was not a problem, as players in allsettings spent a significant percentage of their time on the mathematical activities, i.e. doing word-problems, workingwith numeric clues, and playing the mathgames. The average percentages [ranges] of time on the key activitiesin the controlled school and home setting are shown below. word-prob. clues mathgames School 30[18-46] 17[9-27] 8[0-26] Home 13[6-21] 14[6-23] 28[6-54] click-ons navigating idle School 26[12-41] 18[13-30] 2[0-18] Home 22[9-33] 14[11-18] 9[1-30] The total percent of time spent on math activities wasalmost identical for school and home, averaging 55% in both cases with ranges of 42% to 67% and 42% to 68%, although home players played for substantially longer, with the average time spent at home being 884 minutesover 2 weeks versus 247 minutes over four weeks for the controlled school setting. Our findings from all settingsindicate that the game goal (determining the number of jellybeans) motivates most players to continue to solveword-problems in repeated plays of the game. Click-on activity tended to decline over time, though occasionalsporadic bursts of click-on activity were common. In all settings, there was tremendous diversity in the order ofactivities chosen, as well as the length of time spent on one before moving to the next. One major difference betweencontrolled school and home play was that home players spent a greater percentage of time on the math games. Thisdifference is likely due to the constraints present in the controlled school component (e.g. school norms,preassigned three person teams, restrictions on length of playing time, and lack of choice of playing level). In theclassroom integration component, players' choices of CoF activities responded strongly to both peer and teacherinfluence, resulting in waves of interest in various CoF activities. Teachers exerted influence in a variety of waysincluding holding general class discussions on CoF, asking students to keep journals on what they had discovered, andintroducing specific projects based on CoF. For example, in one project the class collected statistics on the outcomesof three pencil and paper variants of the CoF 9-or-bust mathgame. The class then explored the mathematicalexplanations for their findings, resulting in a nontrivial investigation of probability and combinations. In another,students created their own paper based concentration games exploring visual and symbolic representations ofequivalent fractions, decimals and percentages, mimicking the CoF concentration mathgame. Each such projectcaused a spate of playing of the corresponding CoF activity that lasted several weeks, as well as engaging thestudents in intense discussion and exploration of the underlying mathematics. SUCCESSES General successes in CoF include: players spend a good balance of time between education and entertainment, it isattractive to most girls and boys in the 8-12 age range, and it exposes children to a wide range of content that isrecognizable to them as mathematics. Given our findings on patterns of play, it seems likely that CoF'sattractiveness is related to its wide range of activities, and the freedom to move among them at will. In terms ofspecific educational goals, the clue design seems to work particularly well. Its positive features include the largenumber of possible solutions (15,000) and the types of clues (e.g. "number is less than x", " number does not havean x", and "if number was shared equally among x kids there would be y left over" ). Reflection on the underlyingmathematical concepts (inequalities, number patterns, modular equivalences) is stimulated by several designelements such as giving redundant clues, limiting the number of clues that can be saved, and providing anumber line visualization of the impact of clues to assist in determining the solution (see [2]). We also found that themathgames are attractive in their own right, and provide excellent contexts for classroom projects that engagechildren in deep mathematical thinking (see [1]). PROBLEMS Our main question is to what degree CoF contributes positively (and measurably) to mathematics learning whenit is used in isolation, since in our controlled school sessions the students doing worksheets showed greaterperformance gains than those using CoF, and we found no significant changes in attitudes towards mathematics.There are many factors that could explain this (e.g. the performance tests were almost identical to the worksheets,and the pre-test performance and attitude results were sufficiently high that the results may be subject to ceilingeffects). On the other hand our classroom observations indicate a number of other design elements that may limitCoF's effectiveness as a stand-alone educational tool for learning how to solve word problems, as opposed to onefor practising doing word problems. These elements include the limited number (< 150) and variety (~20) ofword problems to solve, the lack of incentives to move on to the harder levels of the game, the lack of informativehelp when errors are made, and the absence of activities that would engage children in more thoughtful explorationof word problems. ACKNOWLEDGMENTS Our thanks to Karen Bentley, Richard Gibbons, and other UBC students for their help, and to NSERC, EA Canada,BC ASI, and Apple Canada for their financial support. REFERENCES1. Klawe, M. M. & Phillips, E., A classroom study: Electronic games engage children as researchers, inProc. CSCL'95 (Bloomington IN, October 1995). 2. Sedighian, K. & Klawe, M. An Interface Strategy forPromoting Reflective Cognition in Children. Available from 3. Super, D., Westrom, M. & Klawe, M. (1995). DesignIssues Involving Entertainment Click-Ons. Available from