A Japanese View on Games and Learning

Will Games for learning be gospel or Christian rock?

At the first half of the class today, we discussed the current momentum of games for learning or educational games. Guys attended AERA conference last week reported that there are several sessions on games and those were packed with curious audiences. Besides that, we have seen a lot how education people are getting serious about using games for education recently. There were some momentum in the past, but this one seems to be quite huge.

Through the discussion, I thought the situation might be analogous to the Christian people using music to deliver their religious messages. They use music as outreach media as we consider using games to deliver education. Gospel is a successful example of doing this as it became a standard way of delivering message with fun especially for the black people. It also reached out to the outside people who otherwise could have never reached and it is even recognized as a genre of music by general audiences. Maybe even the most religiously conservative people would accept the idea of using gospel music as a mean of delivery.

On the other hand, there is a genre called Christian rock. It is also a genre of music delivering message of Christianity. It literary uses rock music singing religious lyrics, and was quite popular in 80s'. Like gospel, it also reached out the people who could have never reached otherwise. Even for myself, a Christian heavy metal band called Striper was one of my favorite when I was in high school in Japan (their performance throwing bible from the stage was famous at the moment). Though I am not a Christian who have never read bible or listened gospel, the religious messages was delivered with their music to the Japanese high school kid very well. I think there should be a certain power in the Christian rock as well as gospel, and it might reach to the people who don't like gospel as a genre.

However, Christian rock could have never been a mainstream of Christian music. Maybe quite a large portion of people never buy the idea using rock music to deliver the god's message no matter how it turned to be effective. But there are (maybe) still strong funs who are supporting this kind of music tirelessly even though the momentum of Christian rock has long gone. It works for some people, but would never been bought by the majority.

I guess now you see my point. Can games for learning/educational games be a gospel, or Christian rock in this sense when the momentum is gone? Edutainment movement ended up just like Christian rock. How can games be the mainstream like gospel as Christianity delivery media, and also as music genre? If it ended up like Christian rock, it is sort of pathetic. I guess it might end up ilke that if the situation goes as is now, which I think is not much different as edutainment movement. However I still think there is a chance that it could be like gospel if we work hard to come up with innovations.

CBS news coverage on Making History

An high school teacher Dave McDivitt who is actively using a history sim "Making History" in his class describes his experience on his blog.

Dave McDivitt's blog
http://davidmcdivitt.wordpress.com/

He announced that his class was aired on CBS evening news (the video on the page below is the same one as the youtube clip on David's blog.
http://www.cbsnews.com/sections/i_video/main500251.shtml?id=2618924n

EdTech doesn't work??

A research report released by US Department of Education last week seems to make EduTech folks nervous, or get angry. There are lots of discussions on this report going on here and there. Maybe we are also going to discuss on this at the next class.

Effectiveness of Reading and Mathematics Software Products: Findings from the First Student Cohort (National Center for Education Evaluation and Regional Assistance)
http://ies.ed.gov/ncee/pubs/20074005/index.asp

The report basically says that they had a national wide study to assess the impact of education technology on student academic achievement on reading and math, and found that test scores were not significantly higher in classrooms using selected reading and mathematics software products. It also says:
- Though nearly all teachers received training and believed the training prepared them to use the products (software vendors offered training to use the software products), they found they have lower confidence in what they had learned after they began using products in the classroom.
- Technical difficulties using products mostly were minor. The teachers were fine with the product and indicated they would use the product to teach again.
- Students were more likely to engage in individual practice, whereas teachers were more likely to facilitate, rather than lecture.

As a researcher in instructional systems design, I can easily pick several points that EdTech folks might refute what this report says. People might think that the study totally ignores the lessons learned from the previous EdTech studies like, leadership is the key for success to implement technology, and technology integration is not just implement software into classroom, but a systemic change of broader educational systems. There are possible flaws in the research process such as the teacher training before the study might not be sufficient. EdTech news media brought up this sort of points very well.

Major study questions value of school software (e-school news)
http://www.eschoolnews.com/news/showStory.cfm?ArticleID=7011

The research questions like "does educational software work?" is too general questions to be meaningful. As there are good software and bad software products, the results should be different whether you use good software or bad software. Since the study grouped the software and averaged the result, the result can not be better if good and bad software products are averaged. Btw, if you are interested in what kinds of educational software used for the research, Bill MacKenty has compiled the list of the products with links.

Major new edtech report. Some technology doesn’t help math and reading (Bill MacKenty's blog)
http://www.mackenty.org/index.php/site/comments/major_new_edtech_report_some_technology_doesnt_help_math_and_reading/

What can we say from the result? I think there are good/bad things brought up from the report. The good things are:
(1) As the software products used are so called "edutainment" software the teaching approach based on are sort of outdated. If they can at least be a wash to traditional teaching methods, more advanced ones may have better chances to overtake them. This is also claimed by many of the game folks on the serious game list.
(2) The result may bring the data to identify the successful/unsuccessful implementation of the intervention. It should be interesting if the further research to figure out the key factors of success/unsuccess by looking at deeper on the implementation process in each school.

The bad things from this study are:
(1) Though this is also claimed by many others too, the headlines like "No benefit going high-tech for math and reading (CNN)" by news media have a bad influence on general audiences. People might shortsightedly consider that EdTech is no use.

(2) The research may be seen by people as a example of "scientific" research. It is actually a lousy research design as an educational research. Educational research is not like a lab research which the variables can be controlled. Also the research did not a good job on designing the treatment. They can not conclude the software do not have a significant impact on the score since they do not validate if the software used represent the entire group.

All in all, I think this is not a wise way to spend $10M of tax money. It could work as an wake-up call for the EdTech researchers positively, but very expensive one.

Two events on Games for learning

The following events on games for learning/serious games will be taken place soon.

Annenberg Workshop on Games for Learning, Development & Change
Annenberg School for Communication
University of Southern California Los Angeles, California, USA
May 21-22, 2007
Hosted by Annenberg Studies on Computer Games
http://games.uscannenberg.org/AWGHome.php

Seriously Mobile Summit 2007
Date: Friday, 20th April 2007
Location: Reuters Headquarters, Canary Wharf,
Time: 9.30am to 4.30pm
http://www.ambientperformance.com/Information.htm

SGS and GDC Links

Here are some links to the blogs and news websites on Serious Games Summit and GDC. I would update more if I found more of those ones.

GDC: MMOs, past, present and future(Wonderland)
http://www.wonderlandblog.com/wonderland/2007/03/gdc_mmos_past_p.html

Phil Harrison's Keynote(Wonderland)
http://www.wonderlandblog.com/wonderland/2007/03/phil_harrisons_.html

GDC: Miyamoto's keynote
http://www.wonderlandblog.com/wonderland/2007/03/gdc_miyamotos_k.html

GDC 2007 Coverage (Part One of Four) (Blog of Henry Jenkins)
http://www.henryjenkins.org/2007/03/gdc_2007_coverage_part_one_of.html

Raph Koster's Thought on GDC
http://www.raphkoster.com/2007/03/10/thoughts-on-gdc/

GDC 07: Where Game Meets the Web (Raph Koster)
http://www.raphkoster.com/2007/03/10/gdc-07-where-game-meets-the-web/

SGS Recap Day1-2

Testing Assumptions: Creative Approaches to Gathering Evidence of Serious Game Impacts
Speaker: Carrie Heeter (Professor, Michigan State University), Brian Winn (Professor of Game Design, Michigan State University), Richard Van Eck (Associate Professor, University of North Dakota), Caitlin Kelleher (Ph.D. Candidate, Carnegie Mellon), David Williamson Shaffer (Professor, University of Wisconsin)
https://www.cmpevents.com/GD07/a.asp?option=C&V=11&SessID=4418

At the panel, David Schaffer stated that regarding to consider how people think and learn, base model and how you see the world are necessary. Serious games are built based on the model, so the model is very important. Learning occurs explicitly and implicitly, but assessment is mostly explicit. He said that it is like the cart and the horse in a sense that assessment drives instruction and which leads how people judge the content. This is how people focus on the content. However, you cannot test what you cannot teach, and also you can not make a game do everything. Situating assessment in the game design process is one way of improvement. In the game development, design is guided by goals for the experience.

Caitlin Kelleher presented a case of CMU's Alice project. Alice is designed to fit to class time also students can play at home. Recruiting subjects is a tricky part of the design project. If the study is done for school education, it is necessary to consider that education must be accessible to all. The things need to notice include that not all kids are intrinsically motivated by games, and not all kids have equal access to computers. To get kids who are not into computers, they offered a $20 donation to Girl Scout troops for participation in the formative and summative evaluations. Girl scouts are not regular computer users. They also partnered with a local independent school designed to help kids who were failing within the public school system. Advices on formative user tests from her include;
- start small. Early in the process, there is more value in deeply understanding the experience of a few users,
- get every member of the design team to observe users
- find a way to capture users' goals outside of the software if users aren't successful
- As your systems begins to stabilize, include a large numbers of users in evaluations
- record everything users do

She also emphasized on that there are several stuff to do other than play testing such as:
- Sample composition and size
- Formal/informal observation
- Bata testing
- In game data
- Measurement systems
- Facial emotion tracking

Other important issues of conducting evaluation:
- You have to know what you are looking for when you do a controlled study
- Need a time for formative studies
- Only when you have a good hypothesis about what is going to be learned and how does it makes sense to do a summative test

Rick van Eck presented his view on assessment as an ISD researcher who is interested in educational games. His point of view seemed to represent those of instructional designers'. He focused on the importance of outcome analysis and considering it in the context of explicit objectives and assessment. His thought is based on the observable skills as instructional designers do. The assessment process could differ from for those of traditional instructions as they are developed for more linear and passive learning activities. There are usable tool for measurement. Measure of engagement in the design purposes is one example that has evaluation tools (by Patric Dunn http://patrickdunn.squarespace.com/). There are long term concerns like immediate vs. delayed assessment, document validity of models/process.

Other points that are brought up in the session:
- define what the homerun is for your project
- Educate the clients so that they understand good design brings a good learning
- Sponsors like NSF and NIH are looking for reviewers who can evaluate the significance of game-related projects.
- Different expectations for different funding sources

---
Brief summary from other sessions:

Serious Games Next-Gen
Speaker: Peter Smith (Research Faculty, University of Central Florida), Rob Gruhl (Platform Strategist, Microsoft XNA Game Studio Express)

At the session, Rob presented how XNA is useful for game development. 400000 copies of XNA game express were downloaded. More than 80 universities are using it. Peter summarized other next-gen technologies which are possibly usable for serious games development. Tech side of PS3, PSP, DS and Wii are mentioned. The interesting part was using Wiimote to replace expensive lab-use motion controller. Wiimote's technology is not as precise as lab-use one, but could be replaced in many ways. He presented some videos from YouTube that guys using Wiimote to control a huge robot arm.

---
Alcoa SafeDock: A Serious Game Design Case Study by Eben Myers

Eben presented Etcetera's new product for Alcoa. Etcetera is a spin-off company from CMU ETC. The training sim "Safedock" is to train the workers to teach the concept and standards of safety in safe dock. It is another example of serious games for training. Genre wise, it is close to Hazmat:Hotzone, a training sim for firefighters developed by ETC folks.

---
Serious COTS Boggle by Ben Sawyer. Bill Crosbie, Michelle Pun, and Anuja Parikh

The session was the game-based presentation to show the example of Commercial-Off-The-Shelves (COTS) game use for education. Two teams competitively searched the COTS cases available on the internet. There are huge variety of the cases were presented from Myst for creative learning to Never Winternights for sex education. The material presented in the session will be available soon after they compiled all they collected.

SGS Recap Day1-1

Here are some recap from the Serious Games Summit that I attended. As I did not attend the main GDC, I missed a lot of nice sessions like Raph Koster's one and Shigeru Miyamoto's one. As I have found others blogs on those sessions, I would pull together the links for them as well.

---
Serious Games Squared by Ichiro Otobe
https://www.cmpevents.com/GD07/a.asp?option=C&V=11&SessID=4646

The first session was a keynote by Ichiro Otobe of Square Enix.
He explained why Square Enix is interested in Serious Games field.
He pointed out that social recognition on game industry is not high and they suffer from the attacks from mass media. There exists;
- generation gap
- Judge based on the value of old media
- skewed content
Not all new media become mainstream, and games are at the crossroad for it.

The primal reason why they are interested in serious games is because serious games make games be taken seriously.

In Japan, "Serious" comic book culture has elevated it to the mainstream media.
Educational comic books has been received well by teachers and parents for a long time. And nowadays comic books had a huge impact on Japanese culture.
e.g. - Himitsu (secret) series by Gakken (1970s), "Introduction to the Japanese Economy" by Nikken (1986).

Now the games like Brain Age in DS are making top sales in Japan. He noted that there are several challenges to attract the customers. They are concerning they need different skills and business models to address the issue and grow their business in the field.
There are two approaches that they are moving in to SG market. First they started up SG Lab tied up with Gakken, one of the major educational publisher in Japan. SG Lab is focusing on BtoB market, targeting on corporate client and school libraries (Gakken has 24000 elementary schools and 2700 public libraries network). SG Lab is managed as a business unit and does not have development staff internally.

The other attempt is called Project GB, which is developing a DS game to teach game development skills. The project is an internal project at SE. There are many useful skills to learn in the game development process and they teach those skills to kids to nurture a true "game brain". In Japan, game brain is considered as damaged brain which is another burden for media, or fear for unknown by older generations.

The project lower the hurdles for them in several ways. The target customers are the current gamers who they are already handling. The skills taught are what they are really good at and they have plenty of internal resources. Business models already established as the product is for the DS game market. It is also overcome the internal politics as they can convince the opponents saying this is one of the usual DS games that they are familiar with.

It is designed to feature easy entry with depth for those who want;
- access to the low level language
- program logic combining preset units
- edit preset assets, logic

The project is managed to share and collaborate over the net by small team (less than 10 people) so that everyone knows entire project with close communication. They considers that game has benefits in terms of learning/communication such as no manual required, distributed text information, adoption of difficulty to the users, and frequent task and reward cycles.

---
Two reports out:
Speaker: Michelle Lucey-Roper (Learning Technologies Project Manager, Federation of American Scientists), Alicia Sanchez (Research Scientist, Old Dominion University)
https://www.cmpevents.com/GD07/a.asp?option=C&V=11&SessID=4594

Michelle Roper talked about their Summit on Educational Games project, which was sponsored by Federation of America Scientists, ESA, and NSF. She first presented a survey result. It showed what the industry think about educational games market, and about the challenges of R&D for that. The challenge includes; explore features of games important for learning, how to design games to discover positive learning outcomes, tools to create learning games quickly at a low cost, and basic and applied research, technology and prototype development needed.

Alicia Sanchez from Future lab talked about their Teaching with Games project, which was sponsored by EA, Microsoft, Take-two and others. They conducted a survey to UK middle school teachers and students, and also did a case study how teachers design class activities to use commercial entertainment games like The Sims 2 and Rollercoaster tycoon. They found that using games depended more on effective use of teaching skills than game skills. It also is important for teachers to take a central role. Though the games are quite complex and take a while to be familiar with, it is easy to use once familiar with them. To use commercial game in the classroom, teachers should be clear about learning objective. The elements of games could be extracted and used productively in isolation. It is important to know that reflection time should be built into lesson plans. The challenge is making connection between increased motivation to increased learning, and systematic investigation of when authenticity and fidelity are required for specific content areas.