In education research, it’s one thing to show that a learning approach works, but a whole lot harder to understand why. Earlier this year, PhD student Jean Salac and researchers from the CANON Lab directed by associate professor Diana Franklin found that TIPP&SEE, a computer science learning strategy that draws inspiration from reading comprehension, was effective in helping upper elementary students learn and apply basic computing concepts. In their follow-up, the team went under the hood to try to figure out what made TIPP&SEE tick.

Their efforts yielded some surprising results and a Best Paper award at the 2020 International Computing Education Research (ICER) conference, the most prestigious ACM conference on computer science education research. Conference reviewers rewarded the researchers for blending cognitive science and learning science to study what made TIPP&SEE effective in its pilot trial, digging into the “artifacts” of the original study to guide future expansions of the strategy. 

“The original SIGCSE paper provides early evidence that TIPP&SEE is effective. The purpose of the ICER paper is to gain insight into why it might be effective,” said Prof. Franklin.

TIPP&SEE works with the Scratch introductory programming language that is commonly used for early computer science education. But the TIPP&SEE steps build upon that curriculum, inspired by strategies used to teach young children to read and organized around a central protocol of “Use →  Modify → Create.” Students preview a project, look at the code that makes it run, then tinker with the instructions to see how it affects the program. In the final step, they take what they have learned and attempt to build their own code using those principles. 

The original study, conducted in 16 classrooms of students aged 9 and 10, found that students taught with TIPP&SEE exhibited better understanding of computational concepts such as events, sequences, and loops than control students. But that paper, presented at the 2020 SIGCSE meeting in March, only looked at final assessments. For the new ICER study, teams of undergraduates (Zack Crenshaw, Saranya Turimella, Zipporah Klain, and Marco Anaya) over two quarters analyzed additional data collected during the experiment, including worksheets completed by the students and the programming projects they completed.

By studying these “artifacts,” the researchers could gain insight into whether the metacognitive approach of TIPP&SEE successfully promoted behaviors that lead to learning in computer science. 

“Metacognition is the process of thinking about your own thinking,” Salac said. “Think about the steps you take to read an article. As you're reading, you're breaking down the paragraphs into sentences, and then sentences into words and as you assign meaning to individual words, you build up to an understanding of the full sentence. All of this happens quickly and implicitly in an adult reader's head, but to a beginner, these steps can be overwhelming. A similar process happens with reading code, and strategies like TIPP&SEE help make these implicit processes used by experts explicit so beginners can better learn the overarching task.”

When the team looked at student performance on individual projects, they found several encouraging signs that the intervention was inspiring this metacognitive process. Students in TIPP&SEE classrooms completed more project requirements, used more programming elements, and created longer scripts.

However, when the researchers looked at the worksheets that TIPP&SEE students were assigned to complete during instruction, they found a surprising result — or lack thereof. Students reliably completed the Use and Modify sections of the worksheets, but a majority failed to record answers to the third (and what researchers originally thought might be the  most important) step of the protocol, Explore. In addition, there were no correlations between assessments, project attributes, and worksheets. 

“We were very surprised because we saw that the TIPP&SEE students were doing so well with their project requirements,” Salac said. “But we gained some insight when we talked to some of the students — when it came down to it, some students just didn't have time to finish everything. Or maybe they just didn't write it down; they got too busy exploring. But that's something that we have to observe more, to see what students are doing during the projects.”

Despite the lack of correlation, the result will help Salac, Franklin, and CANON Lab design the next phase of TIPP&SEE, for an upcoming study that will expand to multiple school districts. By redesigning class materials and finding new tools for tracking how students work through projects, the next phase can gain deeper understanding on how the learning strategy sticks with students and provides a pathway to computational thinking.

In addition to the award-winning paper, the CANON Lab also presented two additional papers at ICER ‘20 (out of a total of 27 papers). In “An Analysis of Use-Modify-Create Pedagogical Approach’s Success in Balancing Structure and Student Agency,” Franklin and collaborators test whether the protocol at the heart of TIPP&SEE provides a balance of structured and open-ended exploration. In “Exploring Quantum Reversibility with Young Learners,” Franklin’s team studied how to teach a basic concept in quantum science to 8- and 9-year-old students.

Related News

More UChicago CS stories from this research area.
UChicago CS News

UChicago’s Parsl Project Pivots to Sustainability and Community with New Grants

Nov 17, 2022
In the News

Alumnus Pranav Gokhale Named to Crain’s 40 Under 40

Nov 07, 2022
UChicago CS News

Prof. Diana Franklin Discusses Quantum Computing Education on Entangled Things Podcast

Nov 03, 2022
UChicago CS News

CS Students Receive Job Offers at Grace Hopper Celebration

Oct 17, 2022
UChicago CS News

Civic Tech Pioneer James Turk Joins UChicago CS to Teach in MPCS, CAPP

Oct 06, 2022
UChicago CS News

Application Support Program Launched to Help Prospective UChicago CS Students

Sep 30, 2022
UChicago CS News

UChicago/Argonne Computer Scientist Ian Foster Receives ACM/IEEE Ken Kennedy Award

Sep 07, 2022
UChicago CS News

Borja Sotomayor Joins CRA Education Committee as Board Member

Aug 29, 2022
UChicago CS News

High School Students in College Prep Program Visit UChicago CS

Aug 23, 2022
UChicago CS News

UChicago Hosts NSF Workshop on Frontiers of Quantum Advantage

Aug 15, 2022
UChicago CS News

New 2022-23 Faculty Add Expertise in Linguistics, Visualization, Economics, and Data Science Education

Aug 11, 2022
In the News

Bill Fefferman Comments on New Standards for Quantum-Proof Cryptography

Jul 07, 2022
arrow-down-largearrow-left-largearrow-right-large-greyarrow-right-large-yellowarrow-right-largearrow-right-smallbutton-arrowclosedocumentfacebookfacet-arrow-down-whitefacet-arrow-downPage 1CheckedCheckedicon-apple-t5backgroundLayer 1icon-google-t5icon-office365-t5icon-outlook-t5backgroundLayer 1icon-outlookcom-t5backgroundLayer 1icon-yahoo-t5backgroundLayer 1internal-yellowinternalintranetlinkedinlinkoutpauseplaypresentationsearch-bluesearchshareslider-arrow-nextslider-arrow-prevtwittervideoyoutube