Evaluation Report
09/16/02

NASA Ambassador Training
July 15-19, 2002, Sonoma State University


Ted Britton, Associate Director
National Center for Improving Science Education/WestEd

Executive Summary

 

Evaluation Results and Discussion

 
Increased scientific knowledge  
Quality of classroom materials and activities  
Preparation for being a NASA Ambassador  
Conference preparation and execution  
Methods  
Appendices  

Executive Summary


Conclusion. The week-long session addressed well all of its three major goals for training NASA Ambassadors: enhancing their science content knowledge, familiarizing them with NASA-sponsored classroom materials and activities, and preparing them for their roles as NASA Ambassadors. Participants recommended an increased emphasis in future trainings on explicit preparation for their future Ambassador activities.

Background. The participating ten new NASA Ambassadors found the training to be useful and effective, as illustrated by these comments: “Overall, I think this was an excellent workshop”; and “I thought many of the activities will work well in the classroom”. Participants rated the preparation and execution of the agenda and the conference logistics, with minor exceptions, to be very good, as this participant’s remark illustrates: “This is the smoothest week-long workshop that I’ve ever been to”.

Participants did advocate a few small agenda and session changes that would make future trainings even more effective. Participants were appreciative that the training organizers valued their expertise enough to solicit feedback on materials that still are in various stages of the development process. At the same time, however, many participants expressed a desire for slightly less agenda time on instructional materials to make way for increased time on discussing their roles as Ambassadors. Participants valued the content lectures and felt they would achieve deeper understanding of the content if additional, complementary instructional approaches were used, e.g., more time for questions and both large and small group discussions. The evaluator adds that reducing the amount of time for instructional sessions led by the participants themselves could yield time for the things that the participants requested.

1. How much did the training enhance the science content knowledge of participants?

The training increased the scientific knowledge of the participants significantly. After the training, participants knew a little more of the particular content assessed by the SSU content test, as indicated by a 10% change in their pre-post performance responses to the test questions. However, participants clearly valued the broader spectrum of science knowledge that also was covered during the week, finding it to be very relevant to their teaching (m = 4.0 of 5.0). In particular, they rated the two main science lectures by SSU staff as strong favorites among the over 15 sessions in the week’s agenda.

2. How did participants perceive the usefulness of classroom materials/activities for future instruction?

Across all of the 10 activities presented, participants rated their usefulness in the classroom at a value of 3.8 out of 5.0. However, because the participants’ feedback on the materials- or activity-based sessions ranged widely from characteristic to characteristic and session to session, it is important to attend to the results of each session or clusters of sessions rather than overall average. As an example, participants highly rated the activities Build Your Own Galactic Nuclei and Scaling the Spectrum (m = 4.0/4.5 of 5.0, respectively) but weakly rated Spin a Spectrum (3.4).

3. How well did the training prepare participants to be NASA Ambassadors?

Participants clearly felt that the training was useful to their roles as NASA Ambassadors (m = 4.0 of 5.0). However, they recommended three easily achieved enhancements that would increase the training’s effectiveness: regularly draw out more explicit connections between sessions and their implications for Ambassador activities; have participants share their ideas/plans for Ambassador activities; and discuss ideas for Ambassador activities with SSU staff.

4. How did participants perceive the effectiveness of the preparation, organization and delivery of training?

Participants regarded the teaching and presentation strategies delivered by most session leaders as effective (m = 3.9 of 5.0). They would have preferred an opportunity to provide feedback on a draft agenda prior to the training and to receive the final agenda earlier. They had a favorable impression of the preparation that went into the sessions and appreciated the detailed attention that SSU staff paid to conference logistics and participants’ personal needs. Further attention to a few conference arrangements could increase participants’ well-being.


Evaluation Results and Discussion

The report is organized by the four main evaluation questions noted previously in the Executive Summary:

· How much did the training enhance the science content knowledge of participants?

· How did participants perceive the usefulness of classroom materials/activities for future instruction?

· How well did the training prepare participants to be NASA Ambassadors?

· How did participants perceive the effectiveness of the preparation, organization and delivery of training?

These evaluation questions parallel the goals of the training: enhancing participants’ science content knowledge, familiarizing them with NASA-sponsored classroom materials and activities, and gaining their feedback on same, and preparing participants for their roles as NASA Ambassadors.


The overarching purposes of the evaluation were:

· gauge the effectiveness of the training and the SSU-sponsored classroom materials and activities, and

· recommend enhancements of future similar trainings and revisions of classroom materials and activities.


Increased Scientific Knowledge

After the training, participants knew a little more of the particular content assessed by the SSU content test, as indicated by a 10% change in their pre-post performance responses to the test questions (Table 1A). However, participants felt they had gained considerable content knowledge from the sessions that emphasized this goal (Table 1B). Moreover, they clearly valued the broader spectrum of science knowledge covered during the week, finding it to be very relevant to their teaching (m = 4.0 of 5.0) (Table 1B). In particular, they rated the two main science lectures by SSU staff as strong favorites among the over 15 sessions in the week’s agenda (Appendix II, question 108).

From a post-training group interview (See Methods), a majority of participants would have preferred more emphasis in the agenda on content knowledge (ranging from a little more to a lot more). They would reduce the amount of agenda time devoted to classroom materials/activities to provide the additional time needed. Those with weaker scientific backgrounds felt that significant parts of all content lectures were over their heads and would need some discussion with presenters to better understand key points. Participants offered to help each other with science content or anything else over the next few years.

Table 1. Data relevant to science content covered during training

A. Assessment of participants’ science knowledge (pre/post test)

  0.97 average change in number of participants (of 10) answering question correctly
    5 number of questions with gain in number of participants who answered correct
4 number of questions with no change
1 number of questions with loss in number of participants correct
B. Participants’ perception of usefulness of content covered (post and RT surveys)
  2.7 Please rate your knowledge of the covered content prior to this session. (post)
  3.4 Please rate your knowledge of the covered content after this session. (post)
    1 = very weak, 3 = average, 5 = very strong
  3.6 This activity added to my knowledge of space science. (real-time survey)
    1 = strongly disagree, 3 = not sure, 5 = strongly agree
  4.0 How relevant is the covered content for deepening the background knowledge
needed for your current or future teaching? (post)
    1 = not relevant, 3 = somewhat relevant, 5 = very relevant
  3.8 How applicable is the featured content to your classroom?
    1 = not applicable, 3 = somewhat applicable, 5 = very applicable
     
     
     
Source data and additional results available in Appendices 1 and II.

Recommendations

1. Given the SSU staff expertise in science content, they may wish to review the relevant appendices in further detail. For example, what meaning can be ascribed from looking not only at how many participants answered questions correctly, but also at which item distracters they did or did not select (Appendix 1)? What were participants’ relative impressions among the sessions that emphasized increasing content knowledge (Appendices M, II)?

2. Consider adding interactive discussion to the content lectures that would help presenters immediately determine whether participants understood key knowledge and further discuss topics identified as needing more treatment.

3. Look for even more opportunities to explicitly link the presented scientific knowledge to the science that is incorporated in the purpose, design, construction or operation of NASA satellites.

4. Capitalize on participants’ willingness to assist each other with science content knowledge by facilitating communication among them over the coming years.


Quality of Classroom Materials/Activities

Overall, participants’ appraisal of the classroom materials and activities was positive. Table 2 provides feedback on 10 classroom materials that were developed with NASA support. Most responses were between 3.5 and 4.3, where values of 3.1 or higher indicate a positive response.

The GEMS materials were mostly highly rated on a number of dimensions, which is not surprising because they already have been extensively field-tested. The TOPS materials were lowest rated on a number of dimensions in Table 2 and it was participants’ least favorite session in the training (Appendix II, question 109). This also is no surprise because the primary intended audience for the materials is mathematics teachers, while all but one of the Ambassadors are science teachers.

Table 2. Average participant response to NASA-sponsored classroom materials and activities. (5.0 = most favorable response, from RT-surveys).

 

n

develop.

Appropriate

student

interest

student

fun

students

learn

participant

will use

Easy to

implement

Active Galaxy

8

4.1

4.1

4.1

4.6

3.9

3.5

TOPS

10

3.5

3.3

2.9

3.9

2.6

2.6

Spin a Spectrum

5

3.2

4.0

3.8

3.4

3.4

3.5

GEMS

10

4.6

4.6

4.5

4.5

4.3

3.5

Cosmic Survey

10

4.2

3.9

3.8

4.3

4.2

4.3

Local Bubble

8

3.4

3.5

3.5

3.9

3.3

4.0

X-Ray

Eyes

7

3.9

3.6

4.1

4.0

4.0

4.7

Scaling

Spectrum

2*

4.5*

4.5*

4.0*

5.0*

4.5*

4.5*

GPB

10

3.7

4.3

4.5

4.0

3.9

3.9

Anatomy

Blk Hole

10

4.5

4.5

3.9

4.2

4.0

4.0

Average

 

4.0

4.0

3.9

4.2

3.8

3.9

* Results for Scaling the Spectrum are not an accurate representation of all participants’ views because so few participants returned a survey (i.e., n = 2).

Values in bold represent the lowest and highest values for each column. For example, the GEMS activity was perceived as the most developmentally appropriate for students while the Spin a Spectrum activity was perceived as least developmentally appropriate.


Table 3 provides feedback on four additional sessions about classroom activities (versus materials-based sessions in Table 2) led by SSU or others during the training. Participants’ ratings of these activities generally are in the same range as those in Table 2, except they had a slightly negative view of the session on training teachers led by Exploratorium staff (values of 2.9 or less are negative views). Participants had strongly positive impressions of Space Mysteries. Participants favorably rated the tours of scientific facilities in the Bay area that were held on Thursday (Appendix II, questions 48-67).


Table 3. Participant response to other hands-on activities conducted by SSU or outside personnel
(activities not involving draft or final curriculum materials in table 2).
(5.0 = most favorable response, from post-survey).

 

applies to

classroom

Readily

Useable

Likely

to use

T1 training teachers

2.4

2.9

2.4

F1 classrm karaoke

N/a

3.4

3.3

F2 Space Mysteries

N/a

4.4

4.4

Participant present.*

3.8

3.7

3.5

* Refers to activities presented by the participants’ themselves.

Source data and additional results available in Appendix II.

Recommendations

1. Participants felt that many of the products need both small and large revisions, such as clarifying their target audiences and instructional purposes, or enhancing directions and instructional resources for the teacher’s use.

2. Participants recommended that future trainings include more opportunities for participants to discuss/debrief the activities, as they did as the conclusion of the TOPS session.

3. Table 4 provides suggestions that participants made in response to open-ended questions of the RT-survey. Because Table 4 only provides recommendations on major changes needed and/or ones that recur among participants’ answers, SSU may wish to review all of the participants’ responses for details on these changes plus ideas for many small changes.

Table 4. Participant suggestions for enhancing classroom materials. (summarized from open-ended questions of the RT-survey)

Name

Major Recommendations

Active

Galaxy

May not interest advanced students.

TOPS

Appropriate for grades 9-12, but not 8, yet style of graphics may not appeal to older students. Create glastoids more strongly related to the activity and showing relevance.

Spin a

Spectrum

Provide answers and teaching suggestions.

Provide URLs as aid to answering questions.

GEMS

Add an intermediate/translucent option.

Require students to justify predictions.

Add implications for satellite construction.

Cosmic

Survey

Accompanying resources or pointers to resources that would aid students in answering.

Local

Bubble

Address the counterintuitive relationship between density and temperature. Some questions not tied to activity (can be answered independently of the activity).

X-Ray

Eyes

Clarify intended grades for this activity.

Q1 and Q2 need rewording.

Scaling

Spectrum

Add list of materials needed.

GPB

Narrow target audience to grades 10-12 (and perhaps just for advanced courses) because too difficult for grades 7-10. Add background info for teachers, e.g., common misconceptions.

Anatomy

Blk Hole

Math may be too advanced for typical  middle school students. Base activities great, so just add instructional design.

Preparation for Being a NASA Ambassador

Participants perceived the training to be useful in preparing them for their roles as NASA Ambassadors. The average response of participants was a favorable 3.9 to the question asked for every session about whether it “was valuable to my future Ambassador activities”. When asked in Part II about their overall impression of the useful of the training to their Ambassador role, their response was more strongly positive (4.6, Appendix II, question 103).

Ironically, the session most explicitly intended to address the Ambassador role was negatively rated, i.e., participants gave a rating of only 2.3 to the Exploratorium-sponsored session on how to train other teachers (See table 3). The evaluator suspects this is because the presenter spent a small portion of the time overtly discussing the topic (e.g., principle of adult learning and designing and delivering professional development) and instead engaged participants in instructional activities, an element that already was strongly included in other parts of the week’s agenda.

The opinion that participants expressed more strongly than any other topic during the post-training interview was a request to increase the amount of attention in the agenda to explicitly preparing them for the Ambassador role.

Recommendations

1. Participants would have liked an early session to introduce participants, their expertise, and their initial ideas and questions about and plans for being an Ambassador.

2. Regularly discuss connections between sessions and their implications for Ambassador activities.

3. Facilitate sharing ideas/plans for Ambassador activities and hear reactions from SSU staff to their ideas.

Conference Preparation and Execution

Participants had a favorable impression of the preparation that went into the sessions and appreciated the detailed attention that SSU staff paid to conference logistics and participants’ personal needs. Participants very strongly preferred an opportunity to provide feedback on a draft agenda prior to the training and, even more emphatically, to receive the agenda earlier.

Participants regarded the teaching and presentation strategies delivered by most session leaders as effective (m = 3.9 of 5.0, Appendix II), especially those used by SSU senior staff in the science lectures (4.6). The only negative ratings on presentation went to the non-SSU staff who presented TOPS and Training Teachers (2.7 and 2.9, respectively). Participants recommended pedagogical enhancements, such as large and small group discussions to promote deeper understanding the science content presented and solicitation of more and more detailed feedback on instructional materials and activities. Participants greatly appreciated the receptiveness of SSU staff to constructive criticism. For example, not only was every major session evaluated, the SSU also encouraged participants to be very candid in their responses.

Further attention to a few conference arrangements could increase learning conditions by increasing participants’ sense of well-being. For example, participants were critical of the breakfasts (2.4, Appendix II, question 105) and the towels provided by the housing facility (question 107). Overall, however, participants were highly complimentary of the conference logistics and the supportive attitude and responsiveness of the conference staff toward participants’ personal concerns and preferences.

Evaluation Methods

Six sources of evaluation data were collected, as described below. The analysis used the less extensive, qualitative sources of data (4-6) to augment understand derived from the more extensive, quantitative sources of data (1-3). For each of the 15 SSU-organized sessions in the training, Table 5 indicates whether data from the RT and/or Post Survey (sources 2-3) were used to judge its effectiveness. Also for each session, table 5 indicates which of the 4 evaluation questions were analyzed.

1. Pre-post content test. Before and after the training, participants answered ten multiple-choice questions about their science content knowledge.

2. RT-Survey. Participants responded to a short questionnaire immediately after (real-time, RT) each of ten activities that involved using classroom materials developed through NASA-support. Each questionnaire repeated 7 closed-response questions (using a 5-point Likert scale) and 2 open-ended questions.

3. Post-Survey. Immediately after the entire training, participants responded to an extensive questionnaire that investigated their views on all SSU-organized sessions of the training. (The questionnaire inquired only in a general way about the sessions organized by the participants themselves.) Part I of the survey included from 4 to 10 standardized Likert-scale questions about each of 15 individual sessions. Part II asked about the overall program using a combination of 15 closed- or open-response questions.

4. Observation. The evaluator observed 70% of the training.

5. Post group interview. The evaluator interviewed the entire group of participants for 1.0 hour after the training ended.

6. Group discussion. A group discussion was held with the participants by the developer of TOPS after those activities were completed.


Table 5: Sources of Evaluation Data, by Session and by Evaluation Question

Session

Leader

Description

eval.

question

RT-survey

Post-survey

Observ.

M1

LC/PP

science lecture

1,4

 

x

x

M2

LC/PP

SEU activities

2,3,4

X

x

x

T1

TB

training teachers

3,4

 

x

x

T2

TG

GEMS activities

2,3,4

X

x

x

T3

SR

GravityB activity

2,3,4

X

x

x

W1

RM

TOPS activities *

2,3,4

X

x

x

W2

PP

science lecture

1,4

 

x

x

W3

SS/L

blk hole activity

2,3,4

X

x

x

R1

-

SLAC tour

3,4

 

x

 

R2

-

GLAST tour

1,3,4

 

x

 

R3

-

GPB tour

1,3,4

 

x

 

F1

L

Karaoke

2,3,4

 

x

 

F2

PP/TG

Space Mysteries

2,3,4

 

x

 

F3

GG

science lecture

1,3,4

 

x

 

F4

LC

MAP activities

2,3,4

x

x

x

Legend

Session: M = Monday, T = Tuesday, W = Wednesday, R = Thursday, F = Friday
1 = first session, 2 = second session, etc.

Person: LC = Lynn Cominsky, PP = Phil Plait, L = Lynda, TG = Tim Graves,
SS = Sarah Silva, TB = Tory Brady, GG = Gerson Goldhaber

Appendix I. Data and Analysis for Content Knowledge: Pre-posttest assessment of participants’ science knowledge.

 N = 10 participants.

Question (slightly abbreviated)

# pre

# post

# pre-correct

# post-correct

change

1.How does the speed of radio waves compare to that of visible light?

         

A. Radio waves are much slower

4

3

     

B. They both travel at the same speed

6

7

6

7

+1

C. Radio waves are much faster

-

-

     

2.How does the energy in radio wave compare to that in gamma ray?

         

A.Radio waves have much less energy

9

10

9

10

+1

B.Radio waves have the same energy

-

-

     

C.Radio waves have much more energy

1

-

     

3.According to modern ideas, what about center of Universe?

         

A.The Earth is at the center

-

-

     

B.The Sun is at the center

-

-

     

C.The Milky Way is at the center

-

-

     

D.An Unknown distant galaxy is at the center

1

1

     

E.The Universe does not have a center

9

9

9

9

0

4.The hottest stars are what color?

         

A.Blue

7

7

7

7

0

B.Orange

-

-

     

C.Red

1

-

     

D.White

2

3

     

E.Yellow

-

-

     

5.Which list is in order of closest-to-most distant from the Earth?

         

A.satellite, Sun, Moon, Pleiades, Saturn, Alpha Centauri, Andromeda

-

-

     

B.Sun, satellite, Moon, Saturn, Pleiades, Alpha Centauri, Andromeda

-

-

     

C.satellite, Moon, Sun, Saturn, Alpha Centauri, Pleiades, Andromeda

-

-

     

D.Sun, Saturn, satellite, Moon, Alpha Centauri, Andromeda, Pleiades

-

-

     

E.Sun, satellite, Moon, Saturn, Alpha Centauri, Pleiades, Andromeda

10

10

10

10

0

Legend:

Change + = number of additional participants who answered correctly

Change - = number of additional participants who answered incorrectly

Results:

0.97     average change in 10 numbered questions

0.94          average change over  subquestions (of #7,8,10) as well as questions

5          number of questions with gain in number of participants who answered correct

4          number of questions with no change

1          number of questions with loss in number of participants correct

Appendix 2. Average responses for Post-Survey

Monday

Session: M1(am)                 Meet the SEU Missions                Presented by: Lynn, Phil

 2.5      1. Please rate your knowledge of the covered content prior to this session?

1 = very weak              3 = average                        5 = very strong

4.2       2. Please rate your knowledge of the covered content after this session?

1 = very weak              3 = average                        5 = very strong

4.5       3. How relevant is the covered content for deepening the background knowledge needed for your current or future teaching?

1 = not relevant            3 = somewhat relevant             5 = very relevant

3.6       4. How applicable is the feature content to your classroom?

1 = not applicable            3 = somewhat applicable            5 = very applicable

4.9       5. This session should be included in similar programs in the future.

1 = strongly disagree            3 = not sure                  5 strongly agree              

4.5       6. The presenters used effective teaching/presentation strategies.

1 = strongly disagree            3 = not sure                  5 = strongly agree

3.3       7. Enough time was spent learning about the content.

1 = strongly disagree            3 = not sure                  5 = strongly agree

4.5       8. The session was valuable to my future Ambassador activities.

1 = not valuable            3 = somewhat valuable             5 = very valuable


Session: M2(pm)                 Seeing and Exploring the Universe                Presented by: Lynn, Phil, Lynda

Activities in small groups:  a) Swift Spin a Spectrum; b) GLAST—Build your own Active

Galactic Nuclei; c) Chandra—Spectrum Scaling; d) CHIPS local bubble activity; e) XMM Newton

4.7       09. This session should be included in similar programs in the future.

1 = strongly disagree            3 = not sure                  5 strongly agree              

4.4       10. The presenters used effective teaching/presentation strategies.

1 = strongly disagree            3 = not sure                  5 = strongly agree

3.6       11. Enough time was spent learning to use the materials.

1 = strongly disagree            3 = not sure                  5 = strongly agree

4.1       12. The session was valuable to my future Ambassador activities.

1 = not valuable            3 = somewhat valuable             5 = very valuable

Tuesday

Session: T1(am)                  Training other teachers                  Presented by: Tory

2.9       13. How readily useable are the featured materials for your classroom?

1 = prohibitive adaptation needed 3 = doable adaptation needed             5 = use as is

2.4       14. Are you likely to use the featured materials (or adaptations) in your classroom?

1 = wouldn’t use       3 = might use                      5 = definitely use

2.4       15. This session should be included in similar programs in the future.

1 = strongly disagree            3 = not sure                  5 strongly agree              

2.9       16. The presenters used effective teaching/presentation strategies.

1 = strongly disagree            3 = not sure                  5 = strongly agree

3.4       17. Enough time was spent learning about the content.

1 = strongly disagree            3 = not sure                  5 = strongly agree

2.3       18. The session was valuable to my future Ambassador activities.

1 = not valuable            3 = somewhat valuable             5 = very valuable

Session: T2(pm)                  GEMS: Invisible Light Sources and Detectors                Presented by: Tim

4.8       19. This session should be included in similar programs in the future.

1 = strongly disagree            3 = not sure                  5 strongly agree              

4.5       20. The presenters used effective teaching/presentation strategies.

1 = strongly disagree            3 = not sure                  5 = strongly agree

4.6       21. Enough time was spent learning to use the materials.

1 = strongly disagree            3 = not sure                  5 = strongly agree

4.8       22. The session was valuable to my future Ambassador activities.

1 = not valuable            3 = somewhat valuable             5 = very valuable

Session: T3(pm)                  Activities from Gravity Probe-B                 Presented by: Shannon

                                Examining Space-time with Gyroscopes

1.9       23. Please rate your knowledge of the covered content prior to this session?

1 = very weak              3 = average                        5 = very strong

4.2       24. Please rate your knowledge of the covered content after this session?

1 = very weak              3 = average                        5 = very strong

3.8       25 How relevant is the covered content for deepening the background knowledge needed for your current or future teaching?

1 = not relevant            3 = somewhat relevant             5 = very relevant

4.3       26 This session should be included in similar programs in the future.

1 = strongly disagree            3 = not sure                  5 strongly agree              

3.6       27 The presenters used effective teaching/presentation strategies.

1 = strongly disagree            3 = not sure                  5 = strongly agree

4.0       28. Enough time was spent learning about the content.

1 = strongly disagree            3 = not sure                  5 = strongly agree

4.4       29. The session was valuable to my future Ambassador activities.

1 = not valuable            3 = somewhat valuable             5 = very valuable

Wednesday

Session: W1(am)                 GLAST TOPS Learning Activities.                Presented by: Ron                                 

2.7       30. This session should be included in similar programs in the future.

1 = strongly disagree            3 = not sure                  5 strongly agree              

3.6       31. The presenters used effective teaching/presentation strategies.

1 = strongly disagree            3 = not sure                  5 = strongly agree


2.3       32. Enough time was spent learning about the content.

1 = strongly disagree            3 = not sure                  5 = strongly agree

2.4       33. The session was valuable to my future Ambassador activities.

1 = not valuable            3 = somewhat valuable             5 = very valuable

Session: W2(pm)                Gamma-Ray Bursts/Super Novae/ Black Holes                Presented by: Phil

2.4       34. Please rate your knowledge of the covered content prior to this session?

1 = very weak              3 = average                        5 = very strong

4.1       35. Please rate your knowledge of the covered content after this session?

1 = very weak              3 = average                        5 = very strong

4.7       36. How relevant is the covered content for deepening the background knowledge needed for your current or future teaching?

1 = not relevant            3 = somewhat relevant             5 = very relevant

4.5       37. How applicable is the feature content to your classroom?

1 = not applicable            3 = somewhat applicable            5 = very applicable

4.8       38. This session should be included in similar programs in the future.

1 = strongly disagree            3 = not sure                  5 strongly agree              

4.6       39. The presenters used effective teaching/presentation strategies.

1 = strongly disagree            3 = not sure                  5 = strongly agree

4.0       40. Enough time was spent learning about the content.

1 = strongly disagree            3 = not sure                  5 = strongly agree

4.7       41. The session was valuable to my future Ambassador activities.

1 = not valuable            3 = somewhat valuable             5 = very valuable

Session: W3(pm)                Activities: Anatomy of a Black Hole                Presented by: Sarah, Lynda

3.4       42. Please rate your knowledge of the covered content prior to this session?

1 = very weak              3 = average                        5 = very strong

3.9       43. Please rate your knowledge of the covered content after this session?

1 = very weak              3 = average                        5 = very strong

4.6       44. This session should be included in similar programs in the future.

1 = strongly disagree            3 = not sure                  5 strongly agree              


4.0       45. The presenters used effective teaching/presentation strategies.

1 = strongly disagree            3 = not sure                  5 = strongly agree

4.4       46. Enough time was spent learning to use the materials.

1 = strongly disagree            3 = not sure                  5 = strongly agree

4.4       47. The session was valuable to my future Ambassador activities.

1 = not valuable            3 = somewhat valuable             5 = very valuable

Thursday

Session: R1(am)                  Stanford Linear Accelerator Tour                               

4.0       48. How relevant is the covered content for deepening the background knowledge needed for your current or future teaching?

1 = not relevant            3 = somewhat relevant             5 = very relevant

3.4       49. How applicable is the feature content to your classroom?

1 = not applicable            3 = somewhat applicable            5 = very applicable

4.8       50. This session should be included in similar programs in the future.

1 = strongly disagree            3 = not sure                  5 strongly agree              

3.4       51. The presenters used effective teaching/presentation strategies.

1 = strongly disagree            3 = not sure                  5 = strongly agree

3.1       52. Enough time was spent learning about the content.

1 = strongly disagree            3 = not sure                  5 = strongly agree

4.2       53. The session was valuable to my future Ambassador activities.

1 = not valuable            3 = somewhat valuable             5 = very valuable

Session: R2(pm)                  GLAST Tour                                       Presented by: Eduardo

2.9       54. Please rate your knowledge of the covered content prior to this session?

1 = very weak              3 = average                        5 = very strong

3.8       55. Please rate your knowledge of the covered content after this session?

1 = very weak              3 = average                        5 = very strong

3.9       56. How relevant is the covered content for deepening the background knowledge needed for your current or future teaching?

1 = not relevant            3 = somewhat relevant             5 = very relevant

4.7       57. This session should be included in similar programs in the future.

1 = strongly disagree            3 = not sure                  5 strongly agree              

3.2       58. The presenters used effective teaching/presentation strategies.

1 = strongly disagree            3 = not sure                  5 = strongly agree

3.9       59. Enough time was spent learning about the content.

1 = strongly disagree            3 = not sure                  5 = strongly agree

4.3       60. The session was valuable to my future Ambassador activities.

1 = not valuable            3 = somewhat valuable             5 = very valuable

Session: R3(pm)                  GPB Facilities Tour                     Presented by: Jennifer

2.6       61. Please rate your knowledge of the covered content prior to this session?

1 = very weak              3 = average                        5 = very strong

3.9       62. Please rate your knowledge of the covered content after this session?

1 = very weak              3 = average                        5 = very strong

3.9       63. How relevant is the covered content for deepening the background knowledge needed for your current or future teaching?

1 = not relevant            3 = somewhat relevant             5 = very relevant

4.5       64. This session should be included in similar programs in the future.

1 = strongly disagree            3 = not sure                  5 strongly agree              

3.9       65. The presenters used effective teaching/presentation strategies.

1 = strongly disagree            3 = not sure                  5 = strongly agree

3.9       66. Enough time was spent learning about the content.

1 = strongly disagree            3 = not sure                  5 = strongly agree

4.4       67. The session was valuable to my future Ambassador activities.

1 = not valuable            3 = somewhat valuable             5 = very valuable

Friday

Session: F1(am)                   Classroom Karaoke                          Presented by: Lynda

3.4       68. How readily useable are the featured materials for your classroom?

1 = prohibitive adaptation needed 3 = doable adaptation needed             5 = use as is


3.3       69. Are you likely to use the featured materials (or adaptations) in your classroom?

1 = wouldn’t use       3 = might use                      5 = definitely use

4.0       70. This session should be included in similar programs in the future.

1 = strongly disagree            3 = not sure                  5 strongly agree              

3.8       71. The presenters used effective teaching/presentation strategies.

1 = strongly disagree            3 = not sure                  5 = strongly agree

3.9       72. Enough time was spent learning to use the materials.

1 = strongly disagree            3 = not sure                  5 = strongly agree

3.7       73. The session was valuable to my future Ambassador activities.

1 = not valuable            3 = somewhat valuable             5 = very valuable

Session: F2(am)                 Space Mysteries                Demo!                Presented by: Phil, Tim

4.4       74. How readily useable are the featured materials for your classroom?

1 = prohibitive adaptation needed 3 = doable adaptation needed             5 = use as is

4.4       75. Are you likely to use the featured materials (or adaptations) in your classroom?

1 = wouldn’t use       3 = might use                      5 = definitely use

4.6       76. This session should be included in similar programs in the future.

1 = strongly disagree            3 = not sure                  5 strongly agree              

4.2       77. The presenters used effective teaching/presentation strategies.

1 = strongly disagree            3 = not sure                  5 = strongly agree

3.0       78. Enough time was spent learning to use the materials.

1 = strongly disagree            3 = not sure                  5 = strongly agree

4.4       79. The session was valuable to my future Ambassador activities.

1 = not valuable            3 = somewhat valuable             5 = very valuable

Session: F3(pm)                   Dark Matter/Dark Energy/Cosmic                    Presented by: Gerson

2.4       80. Please rate your knowledge of the covered content prior to this session?

1 = very weak              3 = average                        5 = very strong

2.9       81. Please rate your knowledge of the covered content after this session?

1 = very weak              3 = average                        5 = very strong

3.0       82. How relevant is the covered content for deepening the background knowledge needed for your current or future teaching?

1 = not relevant            3 = somewhat relevant             5 = very relevant

3.1       83. How applicable is the feature content to your classroom?

1 = not applicable            3 = somewhat applicable            5 = very applicable

3.8       84. This session should be included in similar programs in the future.

1 = strongly disagree            3 = not sure                  5 strongly agree              

2.7       85. The presenters used effective teaching/presentation strategies.

1 = strongly disagree            3 = not sure                  5 = strongly agree

3.3       86. Enough time was spent learning about the content.

1 = strongly disagree            3 = not sure                  5 = strongly agree

3.4       87. The session was valuable to my future Ambassador activities.

1 = not valuable            3 = somewhat valuable             5 = very valuable

Session: F4(pm)                   Activities: MAP—Geometry in Space             Presented by: Lynn, Phil,

What are your ideas about the Universe?                                          

4.3       88. This session should be included in similar programs in the future.

1 = strongly disagree            3 = not sure                  5 strongly agree              

4.5       89. The presenters used effective teaching/presentation strategies.

1 = strongly disagree            3 = not sure                  5 = strongly agree

4.3       90. Enough time was spent learning to use the materials.

1 = strongly disagree            3 = not sure                  5 = strongly agree

4.8       91. The session was valuable to my future Ambassador activities.

1 = not valuable            3 = somewhat valuable             5 = very valuable

Feedback on Presentations by Ambassadors:

3.4       92. Please rate your knowledge of the covered content prior to these sessions?

1 = very weak              3 = average                        5 = very strong

4.2       93. Please rate your knowledge of the covered content after these sessions?

1 = very weak              3 = average                        5 = very strong

4.4       94. How relevant is the covered content for deepening the background knowledge needed for your current or future teaching?

1 = not relevant            3 = somewhat relevant             5 = very relevant

4.2       95. How applicable is the feature content to your classroom?

1 = not applicable            3 = somewhat applicable            5 = very applicable

4.1       96. How readily useable are the featured materials for your classroom?

1 = prohibitive adaptation needed 3 = doable adaptation needed             5 = use as is

3.8       97. Are you likely to use the featured materials (or adaptations) in your classroom?

1 = wouldn’t use       3 = might use                      5 = definitely use

4.8       98. These sessions should be included in similar programs in the future.

1 = strongly disagree            3 = not sure                  5 strongly agree              

3.9       99. Enough time was spent learning about the content.

1 = strongly disagree            3 = not sure                  5 = strongly agree

3.5       100. Enough time was spent learning to use the materials.

1 = strongly disagree            3 = not sure                  5 = strongly agree

4.0       101. The session was valuable to my future Ambassador activities.

1 = not valuable            3 = somewhat valuable             5 = very valuable


Part II: Feedback on Overall Program

4.4       102. The program was useful for my classroom instruction.

1 = strongly disagree            3 = not sure                  5 = strongly agree

4.6       103. The program helped me prepare for my role as GLAST Ambassador.

1 = strongly disagree            3 = not sure                  5 = strongly agree

4.5       104. The housing met my needs.

1 = strongly disagree            3 = not sure                  5 = strongly agree

2.4       105. The breakfasts met my needs.

1 = strongly disagree            3 = not sure                  5 = strongly agree

3.8       106. The lunches met my needs.

1 = strongly disagree            3 = not sure                  5 = strongly agree

107. What changes would you request in the logistics (housing, food, funding, etc.) of similar programs in the future?

better breakfasts

advance description of housing

better towels at housing facility

108. Please rank order your three favorite sessions.

Gamma Ray Bursts               5

GEMS                                     5

SLAC                                      3

Gravity Probe-B                      3

Seeing/Exploring                      2

GPB tour                                  2

Ambassador presentations            2

Karaoke                                   2

Meet the SEU                          2

TOPS                                      1

Anatomy Black Hole                 1

GLAST tour                             1

Space Mysteries                     1

109. Please rank order your three least favorite sessions.

TOPS                          9

Training teachers            5

Dark Matter                 5

Karaoke                       3

Phys. Science Stds.    1

Online tools                  1

GPB                             1

GLAST Tour                1

110. What session(s), if any, would you keep but only if revised for similar programs in the future? Why, and how would you revise it/them? (If none, please write “none.”)

Ambassador presentations            2            reduce amount of time on this

TOPS                                      2            more science, less math

Gerson                                     1            add some lower level info

Training teachers                        1             more about the advertised topic

111. What session(s), if any, would you discontinue in the future? Why? (If none, please write “none.”)

TOPS                          3

Gerson                         1

Phys. Sci Stds.             1

Karaoke                       1

112. What new content topics, if any, would you add in the future and what kind of session would you like to handle that topic? What would be the purpose of including this (these) new topic(s)? If none, please write “none.”

Sessions about being an Ambassador                 3

113. What new topics about being an Ambassador, if any, would you add in the future and what kind of session would you like to handle that topic? What would be the purpose of including this (these) new topic(s)? If none, please write “none.”

Additional explicit info about SEU missions

Early session to become familiar with each other

Discussion of possible activities, purposes, SSU conditions, examples

114. Other? Are there aspects of the training that you would like to comment upon that were not prompted by this questionnaire? If so, please describe.

Provide clearer goals/expectations of what I should get out of training.

Materials need more instructional design.

Break out science lectures into smaller sessions that include small and large group discussion.

Some of the science was way over my head. Don’t put lectures after lunch when it’s hard to concentrate. Put activities right after lunch.

“Overall, I thought this was an excellent workshop. I loved how cared for I felt through it all.” “Smoothest week-long workshop that I’ve been to.” “I thought it was great!” “The staff was great and many of the activities will work well in the classroom and conventions.” “Everyone at SSU was great – they’ll be a good group of people to work with in the future.”

115. Did any wording or types of questions in this evaluation give you concerns about how your responses will be interpreted?

No       8

Blank   1

Question about time – could mean too much or too little, ambiguous.