Purpose
Guide supporting pre-initiative problem exploration, effective data collection and use, and assessment
Who this is for
When to use
Introduction
Why is this toolkit important?
It provides a structured approach to effectively using data at every stage of a student success initiative. By considering and integrating data from the outset, you ensure that targets are clearly defined, key project indicators (KPIs) are accurately measured, and the initiative’s impacts can be communicated effectively. Additionally, the toolkit supports your initiative assessment and ensures that the data collected is relevant to fulfilling compliance criteria set by your institution or external funders. This proactive approach to data use helps make informed decisions and continuously improves and adapts the initiative to better serve students.
Key actions
How to use this toolkit
This toolkit supports multiple phases of the initiative’s lifespan and is crucial for setting up expectations around data early on to avoid complications and gaps later. It accelerates the initiative process and helps project leads effectively delegate – but not abdicate – responsibility for data use.
If used during initiative planning, the toolkit helps explore and define the problem based on existing data and feedback, leading to the initiative’s Purpose Statement. It also identifies relevant data to track, ensuring a clear understanding of the issues and the metrics needed for success.
As the initiative progresses, the toolkit aids in establishing protocols for consistent and reliable data collection, defining what data should be collected, how often, and how it will be used to assess the initiative’s impact. This structured approach ensures the data gathered is relevant, timely, and aligned with the initiative’s goals.
At the end of a project or at key renewal milestones, the toolkit also serves as a vital resource for communicating the initiative’s impact, helping to clearly present outcomes and guide decisions on future directions.
Tools overview
There are 2 tools in this toolkit. They can be used together or individually.
Related
Next steps
Data Exploration Guide
Why is this tool important?
Already existing information such as anecdotal observations, student feedback, or collected data help you drive problem identification. Approach any problem from all of these angles as you identify and try to deepen your understanding of problems that could be the target of an initiative. After identifying what data to explore, plan for data collection using the Data Collection Guide.
The following considerations are important to help you understand and establish a data baseline. A baseline is a point of reference against which future performance can be measured. It involves collecting and analyzing data at the beginning of a process or project to understand the current state or starting point.
For example, “I noticed students are struggling with…”
These are anecdotal observations from faculty and staff.
For example, “Students are saying…”
Student feedback can be explored from many sources, including:
For example, “I have data that suggests…”
Other preexisting data that builds on or clarifies what you have from sources like survey results. See the Resource—Data Types section below for more information.
Data Types
The following list is a helpful resource for team members to converse with data people (e.g., institutional research, team members). This resource will help you ask practical questions like, “Do we measure this?” “How?” “How often?” “What exactly gets measured?” “Who has this data?” “Who needs it?” “How can I access it?” “Do you see trends?”. Further down the line of data literacy, your questions might be, “How can I see trends? “Is there additional data we need?” and “How/if we cross-reference data to identify intersectional issues?”.
To consider based on your research or initiative focus
Early alerts, midterm grades, term GPA, registration for next term, retention
Participation in events and campus services.
Credit earning by term, ongoing registration, impact of courses on progression (hurdle courses)
Number of pre-requisites
Course attendance, LMS engagement, midterm grades, drop rates, incompletion rates, completion rates
First destination- first full-time position in a field; Salary- differences across a range of variables
Course success rates, persistence rates, transfer rates, completion rates
Disaggregation options to explore or track
Year-defined statuses could be disaggregated in various ways: Common practice is to look at units completed, but one could also look at progress toward completing major requirements.
Factors such as incoming GPA, term GPA, high school courses completed, major/program GPA, Math/English placement
The percentage of students in a course or program who get a D or F grade, withdraw (“W”), or whose progress is recorded as incomplete (“I”) at the student level, by course modality, instructor, participation in certain activities like peer collaborative learning, etc.
Data Exploration Activity
What is driving your problem identification? Try supplementing whatever led you there first with other information to identify a potentially rich target for intervention.
Guiding Questions
Anecdotal observations, student feedback, existing data.
Is there any data that you are missing?
Is there a clear need for intervention or to collect more information?
Template
Project Title
Answer
Answer
Answer
Data Collection Guide
Why is this tool important?
Creating a plan for collecting data to track during an initiative is critical. Lean on your institutional research department for support if you don’t have dedicated data analysis resources on your team.
Before you start, consider the following questions to help you best plan your approach to collecting data throughout your project. These considerations are essential in understanding how and what data will be collected, who will be responsible, and how often.
Reminder! If you haven’t already identified a problem for which to collect data around, consider using the Data Exploration Guide first.
Data Collection Activity
Now that you’ve explored existing data and identified a problem for your initiative, it’s time to develop an approach for collecting data, how you will measure progress, and who the data needs to be communicated to. Reflect on these questions based on your initiative and team capacity before you meet with data people to discuss the specifics.
Guiding Questions
Create a plan for collecting your data.
How will you collect data?
How will you ensure the data collected is disaggregated enough to distinguish historically excluded students?
How frequently does the data need to be updated/collected?
Does it need to be real-time?
Will this data be readily available, or must it be manually tracked?
If manual tracking is required, how will you operationalize it?
Who is responsible for collecting the data?
Where will the data live?
And, who will have access?
If your data are qualitative in nature, who will create the survey/run the focus group/gather feedback, and analyze the results?
Reminder! Disaggregate data to identify equity gaps. For inspiration on what data to disaggregate, see Resource—Data Types—Disaggregation options to explore or track.
How will you set a baseline?
How much granularity do you need in your data?
For example, do you need to track how a student/staff/faculty/etc interacts with your initiative?
What methodology is best given your intended outcomes?
For example, control vs. intervention, pre/post, etc.
How and when do we assess the initiative?
What triggers a reassessment?
What key milestones/checkpoints do we track?
You can use the Communication Plan in the Stakeholder Management Tool to build a more robust stakeholder communication strategy.
Who needs to make decisions based on this data?
Think about who needs to be briefed on data analysis.
How and how often will you share data with key stakeholders?
Template
Project Title
Create a plan for collecting your data.
Answer
Reminder! Disaggregate data to identify equity gaps.
Answer
Think about who needs to be briefed on data and data analysis.
Answer
