Data Cleanup
Find insights hiding in plain sight
Making data usable is the first step to making better decisions
Without usable data, valuable insights can remain hiding in plain sight. Virtually all projects involve data cleanup. Frequently, there is a great deal of clean-up required to generate meaningful understanding.
While data cleanup can be tedious and time-consuming, engaging students in this type of work can be very beneficial. For one, it will not require current employees to choose between managing their daily tasks and this additional responsibility. Second, by doing this type of work, students learn the importance of data quality and will carry this discipline with them to future employers.
Project example
Problem
Over time, a state agency developed multiple sources of information containing facility location data, but there was no unique identifier to combine these sources into a master source of all facility-related information. This process was made even more complicated because some locations shared identification numbers, making it impossible to summarize transactional data by location.
Solution
Student workers at the Institute created unique identifiers within the transactional data by writing a piece of code that used hours and days of operation to distinguish between different locations. This was possible because although locations shared IDs, they would never operate on the same day of the week.
With this combined data repository, it was possible to map all of the locations in a GIS and spatially examine trends.
