For my project I felt as though two surveys would be more useful. Since I needed to collect information about the Martin Aboriginal Education Initiative from a student and staff perspective I felt two different surveys were necessary to analyze information. I proceeded to create two surveys with Survey Monkey and gave them to 4 staff members for proofreading purposes. The only adjustments that were really made from their advice was rewording a few questions and adding a few additional options as answers in the questions that I had not previously thought about beforehand. I did run into a significant issue regarding the survey with the website I chose to use for data collection. The site will not allow you to make more than 10 questions in a survey, export the survey to a .pdf or copy and paste the information in the survey without subscribing to their site with a monthly fee. This became an issue for the purpose of this assignment as I was then unable to remove it from the website once I had created it. Rather than finding a new site and starting all over again and potentially not having the same outline or choices, I chose to take screen captures of the survey to post to my blog. This will appear quite disjointed and would be worse if I posted the original surveys as well as the final drafts. Additionally, I was not aware that I should have posted the original surveys before I had already made the changes to the rough draft. So there are six images that will appear below. The first three pertain to the staff survey and the following three pertain to the student survey. I don't think I will be using survey monkey again in the future unless I need a short survey for online purposes only. What a mess!
Staff Survey:
Student Survey:
We Still Be Bloggin'
Sunday, 13 April 2014
Saturday, 22 March 2014
Assignment #3 Evaluation Worksheet
Planning a Program Evaluation:
Worksheet
Steps in Program Evaluation
Manage
the evaluation
§
Human Subjects
Protection
§
Timeline
§
Responsibilities
§
Budget
|
|
1.
Engage stakeholders |
|||
|
|
|
|
||
2.
Focus |
|
|
|
||
§
Describe
program-logic model
§
Define purpose
§
Determine
use/users
§
Determine key
questions
§
Select
indicators
§
Determine
design
|
3.
Collect data |
|
|
||
§
Identify
sources
§
Select
method(s)
§
Pilot test
§
Set schedule
§
Determine
sample
|
4.
Analyze & interpret |
|
|||
§
Process data
§
Analyze
§
Interpret data
§
What did you
learn?
§
What are the
limitations?
|
5.
Use |
||||
§
Share findings
and lessons learned
§
Use in
decision making
§
Determine next
steps
|
|||||
|
|||||
Standards of
evaluation:
· Utility · Feasibility · Propriety ·Accuracy |
Engage Stakeholders
Who should be involved?
Mount
Royal Collegiate, Classroom teacher, Paul Martin Education Program Rep.,
Aboriginal students, program sponsors
|
How might they be
engaged?
Hearing
about interest level and success of students involved and seeing students
pursue business opportunities in the future.
|
Focus
the Evaluation
What are you going to evaluate? Describe program (logic model).
The
Martin Aboriginal Education Program is a course designed to improve the
graduation rates of Aboriginal students in Canada by introducing them to
entrepreneurship and business opportunities.
They will be able to learn business strategies and methods to give
them an advantage in the work force after high school. If students wish to pursue a business after
high school they will be granted $500 to assist with their startup costs. I
will evaluate if the information they are receiving will prepare them to seek
business opportunities in the future as well as if they are actually planning
on seeking business opportunities after high school. In addition, I will evaluate
if the funding they receive as a program is adequate and allocated
appropriately.
|
What is the purpose of
the evaluation?
The
purpose of the evaluation will be to determine if the program is providing
the necessary skills to Aboriginal youth who are interested in beginning
their own business in the future through a review of their available
resources and engagement level.
|
Who will use the evaluation? How
will they use it?
Who/users
|
How will
they use the information?
|
Classroom
Teacher
|
To
adapt instruction to accommodate student learning and success
|
Martin
Aboriginal Education Program
|
To
review the coordination of funding and available resources to the classroom
teacher
|
Program
Sponsors
|
To
review the funding and practical knowledge provided to students involved in
the program
|
What questions will
the evaluation seek to answer?
Is
the funding model for the program adequate?
|
Are
enough resources available to the students and classroom teacher?
|
Is
the program actually seeing Aboriginal students pursuing business careers in
the future?
|
What information do you need to
answer the questions?
What I
wish to know
|
Indicators
– How will I know it?
|
If
the funding provided to Mount Royal students is adequate.
|
Reviewing
what is being done in the program and how funding is allocated to the
students. Need to determine if there are opportunities being missed due to
insufficient funds.
|
Are
students gaining valuable information from the class
|
Interviewing
or surveying students
|
Are
teachers seeing an interest level from the students
|
Interviewing
or surveying the teachers
|
Are
students choosing to pursue business ventures in the future?
|
Reviewing
statistics on the number of students, that were part of the program, seeking
business opportunities after high school
|
When is the evaluation
needed?
By
the end of the school year to allow for adjustments of the program for the
2014 school year.
|
What evaluation design
will you use?
For
this evaluation I will be using Scriven’s approach of a blended formative and
goal-oriented evaluation. This will
allow for on-going information to help adapt the program in the future but at
the same time evaluating if the program is achieving its intended goals.
|
Collect the information
What
sources of information will you use?
Existing
information:
|
Martin
Aboriginal Education Program website, program brochures, course outlines and
resources
|
People:
|
Teacher,
students, school administrators, MAEP administrators, business mentors
|
Pictorial
records and observations:
|
Surveys,
resource documents, and viewing lessons.
|
What data collection method(s) will
you use?
x
|
Survey
|
x
|
Document review
|
x
|
Interview
|
x
|
Testimonials
|
x
|
Observation
|
|
Expert panel
|
|
Group techniques
|
|
Simulated problems or situations
|
|
Case study
|
|
Journal, log, diary
|
|
Tests
|
|
Unobtrusive measures
|
|
Photos, videos
|
|
Other (list)
|
Instrumentation: What is needed to record
the information?
Online
survey, voice recorder, program documents and resource list, and rooms for
interviews.
|
|
When will you collect data for each
method you’ve chosen?
Method
|
Before
program
|
During
program
|
Immediately
after
|
Later
|
Survey
|
x
|
x
|
|
x
|
Interview
|
x
|
x
|
x
|
|
Observation
|
|
x
|
|
|
Document
Review
|
x
|
x
|
|
|
Testimonials
|
|
x
|
|
x
|
Will a sample be used?
No
|
|
|
Yes
|
x
|
If yes,
describe the procedure you will use.
|
A
sample will be used for interview purposes only. The class size only consists of 10 students
at this point so information should be able to be gathered from each
individual student.
|
Pilot testing: when, where, how?
This
program is already in a pilot testing phase so the evaluation will just add
to the review.
|
Analyze and Interpret
How will the data be analyzed?
Data
analysis methods:
|
Surveys
will be compiled into a spreadsheet for analyzing. Interviews, observations,
and testimonials will be audio recorded and listened to afterward to record
information.
|
Who
responsible:
|
The
evaluator will be responsible for gathering and analyzing the data, as there
will only be a smaller amount of data to collect from the group.
|
How will the information be interpreted—by whom?
Information
will be interpreted by the evaluator with some collaboration with the
classroom teacher.
|
|
What did you learn? What are the limitations?
Limitations
will be getting unbiased and honest responses from the students as they may
feel their answers could impact their mark in the class. Due to the program being early in its
stages it may be difficult to get fully informed answers from staff and
students. There may also be trouble
getting information from parents due to the lower involvement of parents with
this school.
|
|
Use the Information
How will the evaluation be communicated and shared?
To whom
|
When/where/how to present
|
MAEP
|
In the summer through
a written report, statistics, and graphs to allow for any alterations to the
program for the 2014 school year
|
Classroom teacher
|
At the end of the
school year through a collection of statistics and graphs indicating areas of
success and improvement
|
Program sponsors
|
In the summer through
a written report, statistics, and graphs to allow for any alterations to the
program for the 2014 school year.
|
School Administrators
|
At the end of the
school year through a collection of statistics and graphs indicating areas of
success and improvement.
|
Next steps?
Gathering
and analyzing data for the groups listed above in order to help improve and
adjust parts of the program or provide details on the successes of the
program to community members who don’t have the opportunity to regularly hear
about the dynamics of the program.
|
Manage the evaluation Standards
x
|
Human subject’s protection
|
x
|
Utility
|
|
Management chart
|
|
Feasibility
|
|
Timeline
|
|
Propriety
|
x
|
Responsibilities
|
x
|
Accuracy
|
x
|
Budget
|
|
|
Subscribe to:
Posts (Atom)