A new method to measure the tidiness of data repositories and help researchers clean up their “data swamps” took first place in the ACM Student Research Competition for Undergraduates at the 2018 Supercomputing conference in Dallas, TX. The project, conducted by students Luann Jung and Brendan Whitaker with advisors Kyle Chard and Aaron Elmore of UChicago CS, was part of last summer’s BigDataX Research Experiences for Undergraduates (REU) program, held at UChicago and the Illinois Institute of Technology.

Anyone who has used a computer knows how easy it is for data to become disorganized. Without the strictest drag-and-drop discipline, folders quickly become cluttered with files of different formats or subject matter, making it difficult or impossible to find the right file when needed in the future. If this disorder is a problem for personal computers, it is magnified exponentially in the archives of scientific projects that hold terabytes and terabytes of data.

Over the 10 weeks of the BigDataX program, Jung — a first-year student at MIT — and Whitaker — a fourth-year at Ohio State University — worked with a group of UChicago CS researchers seeking solutions for this issue, colloquially known as the “data swamp.” In an CERES-funded project, Chard, Elmore, Ian Foster, Michael Franklin and Blase Ur are developing automated processes that reorganize data repositories or databases and make the information within more reusable and discoverable. But in order to create these improvements, researchers need a way to measure what they’re improving — attaching a number to the dirtiness or cleanliness of a given repository.

“It's an important challenge because a huge portion of scientific research nowadays is heavily reliant on statistical inference from large quantities of data,” Whitaker said. “The work-hour split of data scientist is often said to be nearly 90% preprocessing and 10% inference, training, and testing. It's this ‘heterogeneous’ quality that necessitates such a time sink, and it's my view that quantifying this quality is the first step in creating systems that do it automatically and do it well.”

Measuring Clutter With Clusters

In their paper, “Measuring Swampiness: Quantifying Chaos in Large Heterogeneous Data Repositories,” Jung and Whitaker constructed a parallel pipeline that uses clustering methods to quickly assess a given repository for its tidiness. The pipeline processes text files and tabular data (such as csv or tsv files) and clusters them according to their shared features.

Broadly, the cleanliness score is then calculated from how well those clusters map against file directories in the repository. If a given cluster is heavily represented in a small number of directories and only appears rarely in other locations, that’s considered well-organized. Conversely, a cluster that appears somewhat frequently across many different directories without a significant drop-off between where it is and isn’t present would be considered a sign of disorder.

To test their new score, Jung and Whitaker created synthetic datasets where they could manually shuffle how the files were organized, and also used a real data repository from the Carbon Dioxide Information and Analysis Center. In both evaluations, run using the Chameleon cloud computing testbed, the new cleanliness score outperformed previously-published measures.

Overall, the project combined many of the skills emphasized by the BigDataX program, which is designed to “promote a data-centric view of scientific and technical computing, at the intersection of distributed systems theory and practice.” At Supercomputing, the project beat out dozens of competitors for first place in the undergraduate category, which also qualifies it for the ACM Student Research Competition Grand Finals.

REU and DataSys students at the IEEE/ACM Supercomputing/SC 2018 conference.
REU and DataSys students at the IEEE/ACM Supercomputing/SC 2018 conference.

“I definitely gained experience and practice using machine learning in different contexts. Additionally, the process of writing, organizing, and parallelizing our code together was a good exercise in research collaboration,” Jang said. “Winning the competition was very unexpected but a pleasant surprise.”

Their cleanliness metric will also be used in the ongoing “Data Swamp” project, including as a comparison point for ongoing surveys of how human observers assess the disorganization of repositories.

“I think overall it’s a really hard problem, and one that a lot of organizations deal with. Part of why they won is also their novelty in thinking about how they define their ‘swamp’ metric, which is hard to do given the different types of files found, such as text files, images, and CSVs,” Elmore said. “In just ten weeks, Luann and Brendan provided us with a good foundational component that we can use for future work.”

Applications for the 2019 BigDataX REU program are now open, with a deadline of March 1st, 2019. Join us in Chicago!

Related News

More UChicago CS stories from this research area.

“Machine Learning Foundations Accelerate Innovation and Promote Trustworthiness” by Rebecca Willett

Jan 26, 2024

Nightshade: Data Poisoning to Fight Generative AI with Ben Zhao

Jan 23, 2024
UChicago CS News

Five UChicago CS students named to Siebel Scholars Class of 2024

Oct 02, 2023
In the News

In The News: U.N. Officials Urge Regulation of Artificial Intelligence

"Security Council members said they feared that a new technology might prove a major threat to world peace."
Jul 27, 2023
UChicago CS News

UChicago Computer Scientists Bring in Generative Neural Networks to Stop Real-Time Video From Lagging

Jun 29, 2023
UChicago CS News

UChicago Team Wins The NIH Long COVID Computational Challenge

Jun 28, 2023
UChicago CS News

UChicago Assistant Professor Raul Castro Fernandez Receives 2023 ACM SIGMOD Test-of-Time Award

Jun 27, 2023
UChicago CS News

PhD Student Kevin Bryson Receives NSF Graduate Research Fellowship to Create Equitable Algorithmic Data Tools

Apr 14, 2023
UChicago CS News

Computer Science Displays Catch Attention at MSI’s Annual Robot Block Party

Apr 07, 2023
UChicago CS News

UChicago / School of the Art Institute Class Uses Art to Highlight Data Privacy Dangers

Apr 03, 2023
UChicago CS News

UChicago, Stanford Researchers Explore How Robots and Computers Can Help Strangers Have Meaningful In-Person Conversations

Mar 29, 2023
Students posing at competition
UChicago CS News

UChicago Undergrad Team Places Second Overall In Regionals For World’s Largest Programming Competition

Mar 17, 2023
arrow-down-largearrow-left-largearrow-right-large-greyarrow-right-large-yellowarrow-right-largearrow-right-smallbutton-arrowclosedocumentfacebookfacet-arrow-down-whitefacet-arrow-downPage 1CheckedCheckedicon-apple-t5backgroundLayer 1icon-google-t5icon-office365-t5icon-outlook-t5backgroundLayer 1icon-outlookcom-t5backgroundLayer 1icon-yahoo-t5backgroundLayer 1internal-yellowinternalintranetlinkedinlinkoutpauseplaypresentationsearch-bluesearchshareslider-arrow-nextslider-arrow-prevtwittervideoyoutube