Here we will provide weekly updates on our research progress. In addition to the group blog, each student researcher maintains a personal blog entailing her specific contributions to the research.
At our first meeting, we discussed what a rough timeline for our project would look like and other important logistics. We talked about month by month what we would like to be accomplishing. These projected goals will serve as a guide as we continue working this year. We identified a couple of areas that we should start thinking about now. Such as identifying an audience and contacts that will potentially put us in communication with a middle school we can work with. An important part of this project is figuring out what research has been done and how we can apply that knowledge to our project. After laying out our estimated schedule, we decided that we will be meeting next week to discuss background research on our project.
This week we focused on background information. Individually we will be looking up past research that has been done on main aspects of our project. We are focusing on studies done with middle school aged students as they are our target audience. We will be looking at task completion and color because we feel those are important aspects when we are learning and wanted to find material to support this idea. We also plan on finding articles that show using gaming as a form of instruction. Next week we plan on discussing the articles and how they apply to our project.
This week our group met and discussed the different articles we found. Research does support the idea of using simulations and gaming in the classrom. Often peoplen are hesistant to move away from the traditional way of teaching. But, many are open minded to trying this new approach. Another article showed boys gravitate towards engineering and physical sciences more than girls. Girls who were interested in enginnering contributed that interest to two important factors: early interest in science and participation before 7th grade in science experiences. Next week, we will not be having our regularly scheduled meeting as there is a holiday and Jazzmine and Jillianna will be at the Grace Hopper Celebration!
There was no formal meeting this week as both Jillianna and Jazzmine were attending Grace Hopper. Maria did contact a group at Emmanuel to see about combining research projects.
This week Maria and Jazzmine met with our advisor to ensure we were on track to meet our goals. Jillianna did not attend the meeting due to illness. We adressed some issues and made a list of questions to address for next week. They were as follows:
- Do we want something with a code view or something more like scratch?
- What is the age group we are targeting? Early middle school or early high school?
- If we chose a code view, how feasible is it? What are the pros and cons?
- Create a timeline for the year and hard goals for the next month.
- Accountability with blogs
Adressing the concerns of last week, we have deicided on the following. We will be coding games for Scratch and with a code view- the easier games will be in Scratch and will progress, with difficulty, to code view. We also will be targeting 6th graders for our user group and must find a way to recruit them.
We discussed our goals from last week. Both Jazzmine and Maria are trying to contact others who could help us recruit a testing group. Our goals are to have a basic lesson in Scratch on variables coded by each of us for next week. We will then compare the three games and select the best parts of each. We also will begin applying for the poster session at Tapia 2016.
Our team met this week to discuss the projects we created in scratch. We all decided that we would focus on teaching and using variables. My project focused on teaching basic concepts. I talked about the definiton of variables as well as when a programmer might use them. I then demonstrated through simple buttons on Scratch on variables can be changed and altered throughout a program. Jillianna created a project that focused on users changing code in scratch. Her project required that the user figure out how changing the variables in the program changed the outcome. The third example created by Jazzmine was a more complex variable project. She created a game that used a variable as a way to get a score. Our plan for our next meeting will be to create scratch project discussing if/else statements. We are also planning on working with a group at Simmons called Strong Women Strong girls in two weeks.
The main focus for this week was discussing our if/else programs in scratch. Each person took the topic and created a lesson at varying degrees of difficulty. Maria took the most basic approach. Jillie built on that with more complicated concept and Jazzmine created a more interactive game using if/else. Creating a presentation for the SWSG visit was our second concern. It needs to have enough information with out being boring to the young girls.
This meeting was debrief from the SWSG session we had over the weekend. We demoed a rough version of our Scratch "What is a Variable?" and "If/Else" games. We believe the event was sucessful and we got a better gage as to how children can and will interact with our games in the future.
This week we worked over email to discuss what we will be working on. After working with SWSG, we are going to look at more opportunities to work with them and contact them. We also will be meeting next week to discuss next semester and working on a schedule for completing group blog entries.
Considering this is one of our last meeting before break, we discussed how we want next year to work. Starting a new semester means that we need to find a new time to meet. We are also creating some accountability for keeping the blogs updated. Each person will rotate for project blogs on a google doc. For personal blogs, we will give each other a gentle reminder to up date if one person falls behind.
This week’s project blog is a summary of winter break. Jillie and Jazzmine split parts of the Tapia proposal between each other leaving Maria open to focus on IRB materials given that she has the most experience with it. Jillie advised Jazzmine on ways to present a poster and make the most out of the conference experience.
We are back for the semester. This week we met up to discuss final Tapia proposal. We went over our individual sections and proofread to make it ready to submit. The plan is that Jazzmine will be presenting our research there. We also discussed that Maria will be preparing IRB since we will need to be submitting that soon. We are still hearing back from organizations but have yet to receive a confirmation.
With IRB materials being our top priority, Maria delegated task to each person. Jazzmine is creating the pre and post surveys to determine if the game is in fact teaching the user. Jillie is to complete the satisfaction survey. Maria of course is overseeing everything and in contact with the Simmons’ admin for IRB materials to make sure everything runs smoothly.
This week we have news of a potential test group. Jazzmine is in contact with the Boys and Girls Club of Roxbury with hopes that they will be willing to participate. The final piece missing from our IRB proposal is demographic information from BGC and Jazzmine needs to complete CITI trainings.
This week we talked about what we have done for the IRB so far. Jazzmine and Jillie presented their surveys. Maria and Jillie proofread Jazzmines pre/post assessment to check that the questions were kid friendly. Jazzmine and Maria reviewed Jillies survey as well. We talked about the information we are still waiting from the organization that we might be working with and as the primary contact Jazzmine has been talking with the orgnization about these things.
Contuning to struggle finding a test group, we decided to move forward with creating more levels for our game. We are using a similar tiered system from our previous development. Maria creates an introductory level and a learning level while Jazzmine creates a complex or application leve. Instead of developing levels Jillie will be working on the website to host the games. Additionally, we submitted the IRB proposal. Looking to the future, we are hoping that Jazzmine can continue developing more lessons over the summer and create a complimentaty curriculum using tradional learning to test whether our video game is actually effective at the same level a teacher would be.
This week, we demonstrated what we had worked on for the last week. Jillianna presented her website- we made suggestions as a group and she will be working on a style presentation for next meeting as well. Maria will continue working on IRB forms for another small study involving college students. Jazzmine will work on making more Scratch games, but also putting the search for test groups on the back burner. There will be no meeting next week as it is spring break, so the next meeting will be on the 16th of March.
This week we submitted another IRB to possibly test on college students before getting to children. Again, we reached out to potential sources for a test group. Jillie had jury duty during our meeting but she continued to work on creating a website to host the Scratch games and take in checkpoint information. Maria and Jazzmine have created more levels and are finalizing each level making the directions and game play more clear to younger children.
We are continuing to work on finding a test group of students. We have been approved to work with college students as well. This past week we began writing a new abstract for us to present at the Undergraduate Symposium at Simmons which is a conference meant to present projects that students have been working on this past year. There are two options for the conference either panel or poster. We discussed this afternoon that presenting at a panel would be best for our project.
We have submitted our abstract to the conference and hope to hear about our participation soon. Now that we have our IRB set for college students we are working on letting students know that they are able to participate in our study if they are interested.
We have decided to put all of scratch projects on our own website in order to make it easier for participants to navigate. Jillianna has set up the website and Maria and Jazzmine have focused on adding more instructions explain concepts to the user better. We went through our games again as well as working through our pre/posttest to make sure we were focused on the areas we wanted to in our curriculum.
This week we are focused on getting a test group to run through our curriculum and collecting data. We are continuing to reach out to college students in order to get their feedback on our curriculum as well. We have also added snippets of code in order to be able to show more about the scratch programs that the user will be playing now that it looks sort of different on our website. We created a list of times that we would be available to collect data. We were also accepted for a panel presentation at the Symposium.
Maria is away this week but has been placed in charge of keeping blogs up to date as well as beginning our presentation for Symposium next week. Jazzmine and Jillianna are working on getting college students to participate in our study. We are excited to get their feedback about our curriculum so far and improvements that could be made.
This week was the Undergraduate Symposium at Simmons College. Every year at this conference students from all different departments present projects that they have worked on this past year. This year we were on panel to discuss our project. The panel went well and we were excited that we were able to demonstrate one of the games for those who attended. During our presentation, we talked about the work we have done this past semester and presented the data we collected from college students on our curriculum so far. We were excited to discuss the future of this project and may have a lead for a new test group.
Maria and Jillie have both graduated and Jazzmine will continue working on the project. This week I worked to incoporate some of the feedback from the first bacth of data. Many participants were confused about the mixture of paper and online. I incoporated the paper surveys and quizzes into the lesson so the user does not feel the break. The changes can be seen in the Project Demo tab. Additionally, several of the younger participants wanted a continued story arc in the games. Initially, we wanted the game to have a more "mini-game" feel but kids prefer a storyline. I worked on three different plot-lines. I am not satisified with any of my current games because they are either gender targeted or copies of already popular games.
I started a database of potentional test subjects. Most were middle and elementary schools int nearby cities. Also, I fanned out to some national and local groups that meet with children regularly. While most websites have a "cotact us" page, I search staff directories to find the specific person in charge or programming or voluntering. Using this method I hope to avoid ending up lost in email.
This week, I started by sending emails to a few of the organizations listed. The schools were first to reply. Unfortunately because it is so close to the end of the school year they were unable to participate. But, they all directed me to their summer programs. Also, I found that many high schools were not interested. In more or less words, they felt the project wouldn't add to the students resumes. Taking that into cosideration, I am planning to add another game or informational slide detailing some projects and career paths a computer science degree could bring.
With the constructive feedback from schools, I did some research with kids (my little cousins) in grades 3, 4, 9 and 10. It was clear that the 9 and 10th graders thought the game was too "kiddish" for them but they did provide some good critisms. Mainly, that they didn't know when the games were over so they kept playing them until they were bored. In future interations, there will be a clear stop point to avoid premature boredom. Watching them go through the game, I saw that they only read what was in the scratch games not the html instructions. As a result, they didn't know how to play the games and it caused a good bit of frustration and a key smash at one point. To make sure everything is clear, important points will be displaed in the games and html.
The 3 and 4th graders had a much different experience. It took them significantly longer to go through the entire game but they seemed to geniunely enjoy it. Again, the problem of stoping the games and instructions were a problem. But, afterward they were able to verbally answer questions about Python and answer them correctly. The Google forms were not a hit with them. They had a hard time navigating and it cause some frustrtion. At certain points, I had to define words for them because the games were written for older children
The mini focus group brings up a lot of questions to consider for the remainder of the project.
- Should the target group be younger (3-5 grades)?
- Does the amount of "fun" a game is matter if they are still learning?
- Should the games be more mature or interactive for an older audience?
- Will simplifying the vocabulary lower the quality teachable CS concepts?
This week I made some decisions regarding the questions from last blog. I've decided to stick with the original decision that 3rd grade is too young but I believe that high school age is too old. Since the goal of the project is to get kids interested in programming, it is important for the users to have fun and narrowing the age group down to grades 5th-8th grade will allow for a better learning experience.
Also, in terms of finding a test group. I have had a few programs ask for more information and I am asking for physical meetings instead of email conversations so their is a less chance of getting forgotten about. Speaking of forgetting, my advisor sent 2 potential connects for target groups or girls. And, I haven't reached out yet. But that is first on the list for next week.
I met with a local library and a few camps for kids. The library seemed like a perfect fit because there are already computers and it's a quiet area for testing. After speaking with library staff, they presented the concern of using a publically funded area for research as well as the likely hood of children getting permission slips returned. For the first concern, I emailed some higher ups and ran into the 'no-reply' problem again. For the second concern, I suggested having parents sign when they drop the kids off. But, library staff informed me that most older students (6th grade+) walk or use public transportation to get to the library meaning parents would not be around to sign.
As for the camps, the positives is that permission slips would not be a problem since they are regularly sent home for feild trips and they have access to hundred of children. The 3 major barriers are lack of access to computers. Because the camps met in a high school during the day they only had access to computers for a maximum of an hour. Staffing was another issue. Because I am technically an adult, I need a camp counselor in the room with me and the children. And, the computer rooms are small meaning the conselor to student ratio would fall below their minimum standard. Butm the biggest hurdle was timing. The camps already had much of their programming set months ago and it would be extremely difficult to make changes now.
With that being said, I am going to focus more on improving the demo and place a test group on the back burner.
For this week, I cleaned up the demo. I added directions in the scratch games and the HTML to makes sure everything was clear. After doing some research I realized that the demo was missing a key piece of gamification, rewards. By adding a winning or losing function to the games it is meant to incentivize the user to answer the questions correctly. It also provides a clear stopping point which was a point of feedback from ealrier. The characters now also, direct the user to look at the code snippets.
This was the first week I was abe to collect data. While sitting at the park working on the demo, a young man asked about my project and volunteered to play the games. After a while, a few parents with their children joined too. It was great because I was able to get some data but I also realized a few flaws in the demo. First, the target age group was not happy about having a 'Penny the Programming Penguin' game. They thoguht it was too kiddish and some kids even refused to play. Also, I had the user choose a username and I ran into the problem on them not being unique. Elsa and Dory were two of thr most popular usernames. To fix these problems, I switched back to a non adventure mini games style format and added a random number generator which gives you a number between 0 and 50000. While, it doesn't guranteed uniqueness it is highly unlikely to be assigned the same number twice.
The final week of collecting data had luckily fallen upon a festival at the park so I was able to get much more data than expected. by the end of the week I had a total of 66 completed and 21 incompleted datasets. The large number of incompleted data I belive is due to a design flaw. During the prestest users were able to see how they score and many people that scored 0 points were discouraged and didn't want to continue. After that, I removed that feature from the pre and post test.
This week, I focused on analysing the data for the final report. My statistics experience is very limited leaving a lot of room for research. There were some unique data entries that I had to decided what to do with. Some where people who answered the questions humorously and a few were from young men that decided the post test was a good place to try and flirt. But, my favorite was a 3 year old who had seen their older siblings playing the games and wanted to join. While the toddler didn't understand the concepts he did make the connection between boolean variables and the birthday game. This was a challenge because there is no statistcal way to represet that connection. In the end, I decided to include the datasets but just mark them as wrong answers but to speak more about the unique data in the final report.
Back to the top