Reflection: Evaluating digital services to create the “new normal”
Dorothy Flatman, Dawn Hart and Sarah Frost
When the UK went into lockdown in March 2020, the conversation initially focused on when we would return to 'normal'. As the scale of the pandemic dawned on us, our language shifted. Now we talk about the 'new normal' – and for many early years services that have dramatically changed how they work with and for communities, the question of what this ‘new normal’ should look like now is central.
This has been exemplified by the early years organisations that Dartington and our associates have been fortunate to work with on the Comic Relief Rise and Shine programme. As the Learning Partner for this programme, we’ve supported 21 charitable organisations around the UK working on early years childhood development to reflect on their practice throughout the pandemic and act on the shared learning from evaluation.
This blog reflects on parents’ experiences of services during the pandemic and staff’s hopes and concerns for future delivery. We also share how evaluation has helped organisations understand these, and how it can be used to shape delivery in the future.
Experience for families
Some parents enjoyed the new online sessions – particularly as they could “come as they are”, and have more freedom about when they wanted to engage with services. For others, online sessions proved stressful and difficult - particularly finding space and a convenient time for private conversations away from their children. Group discussions for some families made them feel more exposed by having to talk and share from their own homes.
Projects reported a decline in parents' engagement over time attributed to fatigue in general and with remote contact in particular. It was also noted that many families had become “terrified” about going out and had expressed increased levels of anxiety.
Overall there was consensus that parents' mental health and well-being needs had increased due to real concerns about income and poverty and the challenges of home-schooling. Parents were described by staff as exhausted, "running out of steam" and "just hanging in there".
As restrictions lift, there have been lots of requests for in-person meetings to resume as they were before, in particular from young mums and those in isolated families (such as those in rural communities) but to also keep online sessions as an offer.
Ambivalence from staff
Here, there is a tension for staff. Project teams told us of their mixed feelings about making hybrid (offline and online) services a permanent feature and shared their concern over managing expectations for families as the transition out of lockdown continues.
On one hand, many felt that they should "give [themselves] credit for overcoming a really challenging year" and that they "raised the game [and] stepped out of the comfort zone [providing services] with a lot of quality." There is much to build on here.
However, the move to online delivery felt intense for staff, with many feeling that they are sometimes overcompensating for being at home by putting in longer working days and back-to-back meetings.
Many teams also felt acutely aware of the gap between what they had expected to deliver pre-COVID and what they had actually been able to provide. Even with the additional input reported above, many saw that they had reached fewer families. Comic Relief was a supportive and adaptable funder, but this gap has had an effect on staff.
Using evaluation to improve services
Those we worked with identified a need for a roadmap to guide organisations and practitioners going forward, to avoid over-promising to families, and to build resilient services wherever they are delivered. Many organisations want to continue offering online support but are keen to evaluate if the digital offer is working for families and making a difference.
Dartington’s nine-step guide – Reaching Families Remotely – developed in partnership with early years organisations was shared and discussed with Rise and Shine project teams. The overall goal is to be "clear on purpose" and "flexible on form" which resonated with project teams who shared resources they have used to capture how families feel about online sessions and support in general.
Emojis (to "mix it up and make [evaluation] less dry"), using slido.com, Microsoft forms and Mentimeter were all used to collect data digitally and deliver real-time guidance on whether services were delivering in a way that parents found accessible and useful.
And, as some face-to-face activities resume - including one-to-one sessions, intensive speech and language services, breastfeeding support and other opportunities where families can safely meet - families will be making decisions about what they feel comfortable with and shaping the findings as they do. Continuing evaluation in person will help organisations understand these decisions, and the factors that lie behind them.
The learning and reflection of the Rise and Shine project teams during COVID-19 will shape what they go on to deliver. Building in continued evaluation will allow future decisions to be shaped by the experiences of parents and staff – an invaluable guide as we all continue to navigate the new normal.
This blog was written by the Rise and Shine learning coordinator team commissioned by Comic Relief. They are Dorothy Flatman, Dawn Hart and Sarah Frost.