Student: Caroline Klein, Vassar College

Mentor: Margrethe Horlyck-Romanovsky, Brooklyn College

 

Week 1

After a hectic move-in and learning to use the subway on the first day, I was happy to meet Dr. Wole and all my wonderful cohort members at bowling. Monday was mostly settling in to Hunter and learning about the structure of the program, but later in the week we dove into discussing research conferences and VR, including a brief introduction to ParaView. I also met with my mentor twice to discuss research ideas and develop my project. I spent most of the week researching, brainstorming, and writing my proposal in my room, using the previous REU publications as reference. I also dedicated a significant amount of time to completing my CITI certification to ensure proper research practices.

I am currently planning to develop a virtual buffet environment and design an experiment to see how incorporating gamified elements like nutrient-based point incentives influences people’s food selections in the simulation. It was challenging gauging how much development I could accomplish in 4-5 weeks since I’ve never worked with Unity or VR, but I am open to adjusting my project as needed once I am familiar with the software and have a better sense of what I can accomplish. Although things were a bit unclear at first, I’m excited for the following weeks as I take the next steps with my project and begin learning Unity.

 

Week 2

I spent most of the first half of the week setting up and connecting the MetaQuest headset and learning how to deploy a game from Unity. Kwame was very helpful in this area, and spent time during Wednesday’s class helping me and Amaya get a handle on the basics. I followed some introductory Unity tutorials and played around with the VR development environment on my own as well. For the second half of the week, I completed my literature review and drafted the Introduction, Related Work, and References sections of my research paper. I also completed my CITI certification and tried out Tableau in class on Friday. On my own time, I was excited to attend a women in computing event at Bank of America and get a NY Public Library Card after finally getting our student IDs. 

Next week, my goal is to jump into development and make significant progress on the VR game because the headset logistics took longer than I expected this week and I have not started implementing the actual buffet simulation yet. I aim to finish the game in the next 3 weeks so I have enough time to collect data, analyze results, and finalize the paper. I will also start planning the specifics of the buffet options and gathering information about their nutritional value using the USDA database as my mentor suggested.

 

Week 3

It was challenging resolving runtime errors on my own and piecing together tutorials to develop the features I wanted, but I feel like I made real progress in Unity this week as I began building the VR simulation. I was able to model a simple version of the buffet setting, and implemented movement controls, hand animations, grabbing and ray interactors, text boxes, and updating the overlaid image of a point bar when a food item is selected. These components will lay the foundation of my game. I also conferred with my mentor and explored the available assets on the Unity store to plan the food options, and started writing the methodology section of my research paper. Most of my week was spent on technical development at home, but we also learned more about VR concepts and technologies like immersion, presence, and VMD in class.

 

 

One setback I had this week was that something went wrong when I was working on preventing food items from falling through the floor that made the program very glitchy, and I couldn’t figure out how to fix it. Luckily, I had backed up an early version of the project to GitHub so I was able to retrieve that and redo my work. This was a lifesaver and I will definitely continue to utilize version control on GitHub as I continue to implement the program. However, I am still finding Unity’s errors confusing and sometimes don’t know what is causing them. In the coming week, I hope to expand on my Unity project to have a working prototype of my game with a limited selection of options, before expanding to accommodate all food options and nutrient points in the final version.

 

Week 4

I started off the week with some technical progress as I gathered assets from the Unity Store and arranged them in a 3D scene to make the buffet setting for participants to interact with. One issue I ran into was that we had planned to have several salad options as part of the food selection, but strangely there were no salad prefabs in the asset store. Luckily, I was able to find a lettuce leaf prefab which I used in combination with a bowl, tomato slices, and dumplings (hidden among the leaves to look like chicken) to make the desired salads. Now that the virtual setting is more expansive, I am starting to experience motion sickness when testing the game since I am spending longer in it and moving around more, but it hasn’t been too bad.

 

 

After meeting with my mentor on Tuesday, we decided to amend the methodology and get rid of the two-group design because of the challenge of validating difference results between the groups and the time cost of having many participants since each has to complete the VR simulation. I quickly updated the research paper to reflect this, but plan to rework each section more fully to reflect the change. I feel like I’m having trouble pinning down what type of results and conclusions I want to draw from the experiment now that I’m not comparing how their food choices varied with the gamified intervention, but I hope to meet with my mentor soon to develop this more.

I continued game implementation throughout the week, adding more point bars and the food list to the overlay. I spent the afternoon Thursday trying to integrate a SQLite database, but kept running into errors and ended up unable to successfully add it. I’ve found debugging in Unity very frustrating since it says that I have build errors but gives no indication in the log of where they are coming from. This necessitates building every few minutes to check for errors, but each build takes about 30 seconds to run and you still don’t know exactly what is wrong.

I also prepared a slideshow and demo videos for our midterm presentations on Friday. In this presentation, I briefly went over my proposed idea, literature review, methodology, and technical progress, showcasing the virtual scene and interaction mechanisms. It was also helpful to hear about how everyone else’s projects were developing. I finished up Friday by working on the verbal instructions and Google Forms to give participants before and after the experiment. Next week I hope to combat some of the limitations I identified this week and finish up the technical implementation aspect so the game is ready for user trials.

 

Week 5

This week was heavy in technical implementation as I worked on incorporating feedback from the midterm presentation and finishing up the game. Technical accomplishments include:

• Restructuring the point system code to make it more organized and easier to scale out by associating each food option with a FoodItem object (with attributes for mass, nutrition values, etc.) that I can pass into the add and remove functions.
• Adding ray interactor functionality so that when you hover over the different point bars you can see health information about each nutrient.
• Adding the ability for players to change their food selections and remove items from their plate. This was recommended by my mentor to make participants feel empowered to make informed decisions after seeing the effects of their choices.
• Expanding the existing functionality and details I had prototyped with the burger selection demo to all point bars and food items, including accurate point assignments based on nutrition label data.

I also decided to replace the sushi roll option with a teriyaki salmon and broccoli dish in order to have a dish rich in Vitamin E since that is one of the nutrients I am highlighting. In addition to development in Unity, I worked on planning the pre- and post-surveys for the experiment. My mentor recommended using an established NEMS-P Food Environment survey for reference, so I will plan to incorporate some of those questions and add similarly structured questions about specific nutrients for the knowledge quiz. She also recommended adding an indicator for the recommended daily amount of each nutrient and nutrition labels for each food to the game, so I will look into incorporating these components, too. On campus this week we continued learning about VR topics like GPUs, audio, telepresence, and VR sickness, and played around with Gephi for graph/network visualizations.

 

Week 6

After seeing more of NYC and connecting with the other REU participants on a lunch cruise over the weekend, I jumped into preparing for the research trials. Since I have human subjects testing the game, most of the week was spent preparing for the IRB application. I finished all the required documentation, such as consent forms, and got a faculty member from my home institution to sign off on the Reliance Agreement. I also finalized the Google Form questionnaires and had them approved by my mentor. On the technical side, I added some features to enhance the game experience, such as sounds and an end screen. Adding the background music was simple, but I struggled for a little longer with debugging event-triggered sound effects for food selections.

 

 

I also spent some time reviewing SIGGRAPH Asia XR submissions, which exposed me to some cool research being done. At this point, the VR game and pre- and post-study questionnaire forms are finalized, so I am mostly waiting on IRB approval to start running the study with participants.

 

Week 7

Week 8

At the beginning of the week, I analyzed my user response data, identifying the appropriate statistical tests based on which assumptions were met and following Laerd tutorials to process the data. I used IBM SPSS software to evaluate the significance of the increase in knowledge scores and perceived nutrient importance. Although I found a mean increase in knowledge quiz scores, it wasn’t statistically significant according to the Wilcoxon Signed Ranks Test. However, there was a statistically significant increase in the reported consideration of dietary fiber, lutein and zeaxanthin, and vitamin E after the intervention, though these were based on subjective self-reported scores. I also calculated summary statistics to assess changes in awareness of nutrients and to identify which aspects of the game experience were effective. I updated the results section of my paper to reflect my statistical analysis and wrote the discussion and conclusions sections.

By Wednesday, I had a finished draft of my paper, but I had to revise it because two more people participated in my study, bringing the total to 12 participants. Fortunately, most general results and conclusions remained the same, but I had to spend a few hours redoing the statistical analysis and updating the paper accordingly. The REU group also participated in a Motion Capture event on Wednesday, where different teachers gave short presentations while wearing motion capture technology and we evaluated their performance. It was a long day, but it was exciting participating in an external research study and fun listening to the various talks. At this point, I was in the process of getting feedback from my mentors and revising the paper.

We had our final presentations on Thursday, which consisted of a 10-minute slideshow summarizing our research projects. We presented to our REU peers, mentors, and some Iowa State University REU students. In return, we listened to their research presentations on Friday. The rest of Friday, the group spent the entire day at Hunter finalizing our papers. I created my one-page poster, reviewed submission guidelines, and clarified some details with Dr. Wole. We also completed the end-of-program survey. It was a fun day, as we all worked in the same room, talked over lunch, and helped each other navigate the submission process.

 

 

I’m not fully done with my project at this point because I haven’t submitted yet (almost no one has), but I feel like I am in a good place. My mentor is in Europe right now and has limited availability, but I want to give her a chance to read it over one last time before I submit. I will be submitting a Poster Submission to ACM ISS Poster 2024, which is not due until August 15th, but I hope to send it in early next week. Overall, the Hunter REU program was a very enjoyable experience that I was happy to be a part of. Even if our research didn’t feel the most impactful, I definitely feel more confident with the process of writing and publishing a research paper, which I knew essentially nothing about at the beginning of the program. Beyond that, I loved all my fellow REU participants and couldn’t have asked for a better cohort group. I wish everyone the best of luck in their future endeavors, and hope this blog convinces someone to apply for the REU.