In education research, it’s one thing to show that a learning approach works, but a whole lot harder to understand why. Earlier this year, PhD student Jean Salac and researchers from the CANON Lab directed by associate professor Diana Franklin found that TIPP&SEE, a computer science learning strategy that draws inspiration from reading comprehension, was effective in helping upper elementary students learn and apply basic computing concepts. In their follow-up, the team went under the hood to try to figure out what made TIPP&SEE tick.

Their efforts yielded some surprising results and a Best Paper award at the 2020 International Computing Education Research (ICER) conference, the most prestigious ACM conference on computer science education research. Conference reviewers rewarded the researchers for blending cognitive science and learning science to study what made TIPP&SEE effective in its pilot trial, digging into the “artifacts” of the original study to guide future expansions of the strategy. 

“The original SIGCSE paper provides early evidence that TIPP&SEE is effective. The purpose of the ICER paper is to gain insight into why it might be effective,” said Prof. Franklin.

TIPP&SEE works with the Scratch introductory programming language that is commonly used for early computer science education. But the TIPP&SEE steps build upon that curriculum, inspired by strategies used to teach young children to read and organized around a central protocol of “Use →  Modify → Create.” Students preview a project, look at the code that makes it run, then tinker with the instructions to see how it affects the program. In the final step, they take what they have learned and attempt to build their own code using those principles. 

The original study, conducted in 16 classrooms of students aged 9 and 10, found that students taught with TIPP&SEE exhibited better understanding of computational concepts such as events, sequences, and loops than control students. But that paper, presented at the 2020 SIGCSE meeting in March, only looked at final assessments. For the new ICER study, teams of undergraduates (Zack Crenshaw, Saranya Turimella, Zipporah Klain, and Marco Anaya) over two quarters analyzed additional data collected during the experiment, including worksheets completed by the students and the programming projects they completed.

By studying these “artifacts,” the researchers could gain insight into whether the metacognitive approach of TIPP&SEE successfully promoted behaviors that lead to learning in computer science. 

“Metacognition is the process of thinking about your own thinking,” Salac said. “Think about the steps you take to read an article. As you're reading, you're breaking down the paragraphs into sentences, and then sentences into words and as you assign meaning to individual words, you build up to an understanding of the full sentence. All of this happens quickly and implicitly in an adult reader's head, but to a beginner, these steps can be overwhelming. A similar process happens with reading code, and strategies like TIPP&SEE help make these implicit processes used by experts explicit so beginners can better learn the overarching task.”

When the team looked at student performance on individual projects, they found several encouraging signs that the intervention was inspiring this metacognitive process. Students in TIPP&SEE classrooms completed more project requirements, used more programming elements, and created longer scripts.

However, when the researchers looked at the worksheets that TIPP&SEE students were assigned to complete during instruction, they found a surprising result — or lack thereof. Students reliably completed the Use and Modify sections of the worksheets, but a majority failed to record answers to the third (and what researchers originally thought might be the  most important) step of the protocol, Explore. In addition, there were no correlations between assessments, project attributes, and worksheets. 

“We were very surprised because we saw that the TIPP&SEE students were doing so well with their project requirements,” Salac said. “But we gained some insight when we talked to some of the students — when it came down to it, some students just didn't have time to finish everything. Or maybe they just didn't write it down; they got too busy exploring. But that's something that we have to observe more, to see what students are doing during the projects.”

Despite the lack of correlation, the result will help Salac, Franklin, and CANON Lab design the next phase of TIPP&SEE, for an upcoming study that will expand to multiple school districts. By redesigning class materials and finding new tools for tracking how students work through projects, the next phase can gain deeper understanding on how the learning strategy sticks with students and provides a pathway to computational thinking.

In addition to the award-winning paper, the CANON Lab also presented two additional papers at ICER ‘20 (out of a total of 27 papers). In “An Analysis of Use-Modify-Create Pedagogical Approach’s Success in Balancing Structure and Student Agency,” Franklin and collaborators test whether the protocol at the heart of TIPP&SEE provides a balance of structured and open-ended exploration. In “Exploring Quantum Reversibility with Young Learners,” Franklin’s team studied how to teach a basic concept in quantum science to 8- and 9-year-old students.

Related News

More UChicago CS stories from this research area.
UChicago CS News

UChicago London Colloquium Features Data Science, Quantum Research

Jul 01, 2022
UChicago CS News

EPiQC Post-Doc Pens Op-Ed on Potential of Quantum Computing for Chemistry

Jun 24, 2022
UChicago CS News

Faculty Bill Fefferman and Chenhao Tan Receive Google Research Scholar Awards

Jun 21, 2022
Video

Data Science Institute Summit

Jun 15, 2022
UChicago CS News

DSI Summer Lab Returns In-Person With 49 Students From Across the U.S.

Jun 14, 2022
UChicago CS News

UChicago CS Spinout Super.tech Acquired by Quantum Ecosystem Leader ColdQuanta

May 10, 2022
UChicago CS News

First-Year PhD Student Co-Authors Outstanding Paper Award Winner at TQC 2022

Apr 28, 2022
UChicago CS News

UChicago CS Labs Join Museum of Science & Industry For Robot Block Party

Apr 20, 2022
UChicago CS News

Super.tech/EPiQC Quantum Computing Benchmark Receives Best Paper Award

Apr 14, 2022
UChicago CS News

Concurrency Bug Research by Prof. Shan Lu Receives ASPLOS Influential Paper Award

Mar 29, 2022
UChicago CS News

Professor Andrew Chien Authors New Textbook, “Computer Architecture for Scientists”

Mar 16, 2022
In the News

Quanta Magazine Features Prof. Bill Fefferman’s Work on Quantum Algorithms

Jan 20, 2022
arrow-down-largearrow-left-largearrow-right-large-greyarrow-right-large-yellowarrow-right-largearrow-right-smallbutton-arrowclosedocumentfacebookfacet-arrow-down-whitefacet-arrow-downPage 1CheckedCheckedicon-apple-t5backgroundLayer 1icon-google-t5icon-office365-t5icon-outlook-t5backgroundLayer 1icon-outlookcom-t5backgroundLayer 1icon-yahoo-t5backgroundLayer 1internal-yellowinternalintranetlinkedinlinkoutpauseplaypresentationsearch-bluesearchshareslider-arrow-nextslider-arrow-prevtwittervideoyoutube