In education research, it’s one thing to show that a learning approach works, but a whole lot harder to understand why. Earlier this year, PhD student Jean Salac and researchers from the CANON Lab directed by associate professor Diana Franklin found that TIPP&SEE, a computer science learning strategy that draws inspiration from reading comprehension, was effective in helping upper elementary students learn and apply basic computing concepts. In their follow-up, the team went under the hood to try to figure out what made TIPP&SEE tick.

Their efforts yielded some surprising results and a Best Paper award at the 2020 International Computing Education Research (ICER) conference, the most prestigious ACM conference on computer science education research. Conference reviewers rewarded the researchers for blending cognitive science and learning science to study what made TIPP&SEE effective in its pilot trial, digging into the “artifacts” of the original study to guide future expansions of the strategy. 

“The original SIGCSE paper provides early evidence that TIPP&SEE is effective. The purpose of the ICER paper is to gain insight into why it might be effective,” said Prof. Franklin.

TIPP&SEE works with the Scratch introductory programming language that is commonly used for early computer science education. But the TIPP&SEE steps build upon that curriculum, inspired by strategies used to teach young children to read and organized around a central protocol of “Use →  Modify → Create.” Students preview a project, look at the code that makes it run, then tinker with the instructions to see how it affects the program. In the final step, they take what they have learned and attempt to build their own code using those principles. 

The original study, conducted in 16 classrooms of students aged 9 and 10, found that students taught with TIPP&SEE exhibited better understanding of computational concepts such as events, sequences, and loops than control students. But that paper, presented at the 2020 SIGCSE meeting in March, only looked at final assessments. For the new ICER study, teams of undergraduates (Zack Crenshaw, Saranya Turimella, Zipporah Klain, and Marco Anaya) over two quarters analyzed additional data collected during the experiment, including worksheets completed by the students and the programming projects they completed.

By studying these “artifacts,” the researchers could gain insight into whether the metacognitive approach of TIPP&SEE successfully promoted behaviors that lead to learning in computer science. 

“Metacognition is the process of thinking about your own thinking,” Salac said. “Think about the steps you take to read an article. As you're reading, you're breaking down the paragraphs into sentences, and then sentences into words and as you assign meaning to individual words, you build up to an understanding of the full sentence. All of this happens quickly and implicitly in an adult reader's head, but to a beginner, these steps can be overwhelming. A similar process happens with reading code, and strategies like TIPP&SEE help make these implicit processes used by experts explicit so beginners can better learn the overarching task.”

When the team looked at student performance on individual projects, they found several encouraging signs that the intervention was inspiring this metacognitive process. Students in TIPP&SEE classrooms completed more project requirements, used more programming elements, and created longer scripts.

However, when the researchers looked at the worksheets that TIPP&SEE students were assigned to complete during instruction, they found a surprising result — or lack thereof. Students reliably completed the Use and Modify sections of the worksheets, but a majority failed to record answers to the third (and what researchers originally thought might be the  most important) step of the protocol, Explore. In addition, there were no correlations between assessments, project attributes, and worksheets. 

“We were very surprised because we saw that the TIPP&SEE students were doing so well with their project requirements,” Salac said. “But we gained some insight when we talked to some of the students — when it came down to it, some students just didn't have time to finish everything. Or maybe they just didn't write it down; they got too busy exploring. But that's something that we have to observe more, to see what students are doing during the projects.”

Despite the lack of correlation, the result will help Salac, Franklin, and CANON Lab design the next phase of TIPP&SEE, for an upcoming study that will expand to multiple school districts. By redesigning class materials and finding new tools for tracking how students work through projects, the next phase can gain deeper understanding on how the learning strategy sticks with students and provides a pathway to computational thinking.

In addition to the award-winning paper, the CANON Lab also presented two additional papers at ICER ‘20 (out of a total of 27 papers). In “An Analysis of Use-Modify-Create Pedagogical Approach’s Success in Balancing Structure and Student Agency,” Franklin and collaborators test whether the protocol at the heart of TIPP&SEE provides a balance of structured and open-ended exploration. In “Exploring Quantum Reversibility with Young Learners,” Franklin’s team studied how to teach a basic concept in quantum science to 8- and 9-year-old students.

Related News

More UChicago CS stories from this research area.
Card Image e1bf793
Card Image
UChicago CS News

UChicago Scientists Make New Discovery Proving Entanglement Is Responsible for Computational Hardness In Quantum Systems

Card Image 8a10135
Card Image
UChicago CS News

Computer Science Displays Catch Attention at MSI’s Annual Robot Block Party

Card Image 0ea4f6e
Card Image
UChicago CS News

Virtual Bakery Game Serves Up Both Cupcakes and Quantum Concepts For K-12 Students

Card Image dcddf2b
Card Image
UChicago CS News

UChicago Undergrad Team Places Second Overall In Regionals For World’s Largest Programming Competition

Card Image 2bdf2ec
Card Image
UChicago CS News

Student Spotlight: Gabi Garcia’s Bridge Between CS and Classics

Card Image 754d9d9
Card Image
UChicago CS News

High School Students Find Their Place in Computing Through Wearables Workshop

Card Image 4171fd6
Card Image
UChicago CS News

Assistant Professor Robert Rand Receives Air Force Young Investigator Grant

Card Image f1a8cf2
Card Image
UChicago CS News

Prof. Fred Chong Reappointed to National Quantum Initiative Advisory Committee

Card Image 2372712
Card Image
UChicago CS News

Professor Fred Chong Named IEEE Fellow

Card Image f874a63
Card Image
UChicago CS News

Associate Professor Diana Franklin Named ACM Distinguished Member

Card Image 4ddedac
Card Image
UChicago CS News

The Computing Pipeline: A Foundation for Diversifying Computer Science

Card Image 9b385b1
Card Image
UChicago CS News

UChicago’s Parsl Project Pivots to Sustainability and Community with New Grants

arrow-down-largearrow-left-largearrow-right-large-greyarrow-right-large-yellowarrow-right-largearrow-right-smallbutton-arrowclosedocumentfacebookfacet-arrow-down-whitefacet-arrow-downPage 1CheckedCheckedicon-apple-t5backgroundLayer 1icon-google-t5icon-office365-t5icon-outlook-t5backgroundLayer 1icon-outlookcom-t5backgroundLayer 1icon-yahoo-t5backgroundLayer 1internal-yellowinternalintranetlinkedinlinkoutpauseplaypresentationsearch-bluesearchshareslider-arrow-nextslider-arrow-prevtwittervideoyoutube