In September 2014, NYMAZ and delivery partners Harrogate International Festivals and Musicport implemented a new evaluation method for two Early Years projects, Musical Mums (Harrogate) and Music Tots (Whitby), funded by the National Foundation for Youth Music. The new method has enabled us to track the progress of participants in a range of areas of development, including vocal participation, interaction with peers, understanding of repertoire, and relationships with parents/carers.
The method used questionnaires completed by the music leader or support staff, based on observations of the children.
1. A baseline questionnaire was completed for each child at the beginning of the evaluation period (or when they joined the project).
2. The same questionnaire was completed at regular intervals throughout the project.
Statistics drawn up automatically by our online monitoring and evaluation system, called Substance Views, tell us how many children were assessed at each level for each question, and we have been able to compare the results of the baseline questionnaires to assessments throughout the year. We have now summarised some of the findings of the first two terms of data.
Statement 1 on the questionnaire was “Engages in active listening within the session. (What to look for - eye contact, facial expressions, emotional responses)”. Each child was rated on a five point scale, from ‘strongly agree’ to ‘strongly disagree’.
Figure 1 (n= sample size)
As you can see from ‘Figure 1’ above, the baseline results showed that many of the children were already engaging in active listening, with the ‘agree’ or ‘strongly agree’ ratings given to 60% of participants. But in comparison, by the end of Term 2, 100% of participants had those ratings, suggesting that those children not showing signs of engagement at the start of the period had increased their engagement, including eye contact and facial expressions, after two terms of weekly activity.
Statement 2 was “Engages in vocal participation. (What to look for - mouth movements, babble, cooing etc. in older children - more general observations of singing or speaking within the framework of the song/session)”.
Figure 2 (n= sample size)
Again, the results show an overall improvement in the period, as you can see in ‘Figure 2’ above. At the point of the baseline assessment, there was a range of responses, with the average response across both projects just above ‘disagree’. In contrast, the ratings ‘strongly agree’ and ‘agree’ accounted for nearly 70% of participants by the end of the period.
The above two results helped us to evidence one of the outcomes we hoped to achieve: “To improve the communication, language and literacy development of young children at higher risk of delay”.
Another intended outcome, “To improve the personal, social and emotional development of young children at higher risk of delay”, was measured using a number of other statements. For example, statement 10 on the questionnaire was “Interacts positively with other children within the group”. At the baseline point, the average response to this question was ‘neither agree nor disagree’. By the end of term 2, the average was between ‘agree’ and ‘strongly agree’, with around 55% of the sample given the ‘strongly agree’ rating. See ‘Figure 3’ below.
Figure 3 (n= sample size)
Improved positive interaction did not only apply between the children, but between child and parent/carer, too. In response to the statement “Have you witnessed a positive, engaging relationship between the participant and his or her parent/carer?” there was a small but notable improvement between the baseline and final assessments. Initially, responses were ‘neither agree nor disagree’ for 4 participants, and ‘strongly disagree’ for 1 participant, out of the sample of 21. By the final assessment, all responses fell under ‘strongly agree’ or ‘agree’.
Figure 4 (n= sample size)
The variance in sample size between the baseline assessment and final assessment must be noted, and this was due to a number of the participants assessed at baseline not attending the project for the full period. We recognise that this may have had an impact on the results.