Attracting Minorities to Geosciences Through Involved Digital Story Telling
Assessment

AMIDST: Evaluation and Assessment

Evaluation, as used in the context of the AMIDST project, refers to the ongoing process of measuring progress against expected outcomes at all stages of the project. As the focus is on timely qualitative and quantitative measures, achieved by comparing data to a standard or a stated objective for the purpose of judging worth or quality, evaluation is primarily formative in nature. However, we do have summative evaluation as well.

Assessment, as used in the context of the AMIDST project, is a summative or final measurement of activities or competencies against established criteria. It refers to the collection of data to describe or better understand an issue.

For more information on education evaluation and assessment, refer to the ESSE Evaluation Toolkit. Evaluation and assessment for the AMIDST project was carried out by Cheryl Cooper (Alaskan resident) in consultation with Roger Levine (NSF Education Project Evaluator).

Evaluation / Assessment Tools

We first used diagnostic evaluation surveys to gauge the content knowledge, attitudes, perceptions and expectations of the participating students and teachers. This helped to get a good sense of how to tailor or content material to meet the needs of our target audience.

We later used the same set of questions in a summative assessment surveys to assess the change in content knowledge, and also in the participants' belief system and perceptions. Besides the pre-post tests, our evaluator also ran a mid-term moderated focus group meeting to provide feedback to the PIs on what is working and where corrective measures need to be taken in project implementation. She also made a summative assessment using feedback forms and exit interviews that she either recorded or transcribed.

Summary of Significant Results

Students, from Fairbanks and Nome alike, showed statistically significant increase in geoscience content knowledge and reflected a favorable attitude toward science. There was fluctuation in attitude towards storytelling. Some students did not like the 'actual writing' of the story, while some others were shy and were not very excited about performing the story. Teachers were exposed to geosceince concepts, media literacy, integrating story telling in the curricula, cultural perspectives, and digital techniques, all in a very short time. This was a bit too intense. Teachers showed the greatest increase in the area of knowledge about implementing digital story telling in a classroom environment. Though teachers and project scientists were overall happy with the learning experience, our sample size for teacher groups was very small to be able to confidently claim the statistical significance of the teacher training results.

Moderated focus group with the students involved a voting by show-of-hands on different elements of the project.
  • Field trips rated the heighest on the popularity scale
  • Writing the story was the least favorite part (developing the story, on the contrary, was fun for some)

Selected Participant Opinions

Opinion of Elders

Opinion of Teachers

Opinion of Teachers

Opinion of Students

Opinion of Scientists

Opinion of Scientists

Lessons Learned

Though some of the points presented here may intuitively appear as obvious, this is a valuable summary of our learning experience in implementing the NSF OEDG Track 1 (proof of concept) AMIDST project
  • In Alaska, especially in rural Alaska, the class sizes are small. In one class students can have starkingly different intellectual and cognitive skills. Coupled with the fact that students come from families with high economic diversity and varying levels of parent involvement, the class dynamics can be very different from year-to-year. The same project / activity can therefore, show contrasting success and results in two successive years.
  • In implementing a technology driven project/curricula it is far more pragmatic and sustainable to work with the schools existing technology set-up and the school districts technology instructor, than to rely on external help to meet the technology needs.
  • There is only limited time and flexibility the school system can offer to implement an additional element/project in their existing set-up, even if you have made all efforts to weave the project intimately with their existing curricula. A digital story telling project, which also has elements of geoscience content and native knowledge, is quite time-intensive. It is better to either implement it as a summer program, or to spread it over two semesters than crunch everything in one semester.
  • Parents of children in larger cities are more open to giving permission for their children's participation and photo release than parents of children in smaller places and rural settings. It is important for the PI's to talk to the relevant authorities at their home institutions and urge them to keep the parent permission and media release permission forms as simple as possible.
  • Some children are particularly shy to be in front of the camera. Some, especially those who've just had braces or are going through a voice change phase, are very conscious of ther voices. If possible, children should be given the choice to select between green screen story telling and computer-based story telling.