HowwedevelopedChalkboardGuides

Ee-Reh Owo, Schools Director at Justice Rising shares the key learnings from the development of the prize-winning Chalkboard Guides.

When we first started talking about Chalkboard Guides at Justice Rising, we weren’t thinking about iterative design. We were just trying to solve a problem: how can we use teacher guides to boost teacher effectiveness in our network?

Teacher guides are not new, but their implementation has relied until now on complementary workbooks, extensive training and follow up. Distribution of individual student workbooks and access to rebel-held areas for follow up presented difficult logistical challenges for our schools, which provide education for children growing up amidst protracted conflict in eastern DRC (Democratic Republic of the Congo). These challenges also drove up the cost of the intervention, so we needed to find a different way.

We used evidence from cognitive and behavioural science to inform the design of all-in-one, highly intuitive and therefore more adoptable structured lesson guides. Our Chalkboard Guides harness the ubiquity of the board as a teaching tool to support teachers serving the most marginalised communities to deliver quality education. Each guide provides teachers with an objective for the lesson, teacher and student activities and a visual representation of the board work required. 

Upon entering the Jacobs Foundation and MIT Solveathon in April 2021 with our idea, we came away with an Audience Award of 20,000CHF to prototype them and, with iterative design, an approach that would make Chalkboard Guides even more intuitive, by putting teachers at the centre of the design process.

This was our first foray into prototyping a product, but over the course of three field-tests spanning literacy and numeracy, in two countries, two grades and three languages, it has been a rich learning experience (and we have a prototype!). Here, I want to share why and how we set about the iterative-design process.

Why iterative design? With Chalkboard Guides, our aim is to create guides that teachers love - because they are intuitive, make teachers feel more effective quickly and ease teacher workload. Iterative design gives us a process to test and refine Chalkboard Guides to reach that goal. Most importantly, using iterative design we were able to put teachers at the centre of the design process. We listened to their feedback and monitored their interaction with the initial versions of Chalkboard Guides, incorporating their suggestions to reach a viable product. Listening to teachers’ voices and respecting their agency (by making the guides optional) put the onus on our team to create intuitive, highly adoptable guides that teachers wanted to use.

How did we go about the iterative design process? Iteratively! We had a go, then we tried again, and again, monitoring, learning and improving as we went.

This was the process that we followed each time, making adjustments from field test to field test. For example, during the first field test we didn’t leave long enough for design and ended up rushing the preparation for the teacher workshop (step 4), so we learnt from it for the next field test. Teachers were enthusiastic about the guides right from the start, but we were unsure about the robustness of our indicators for fidelity and adoption, so we reworked those indicators for the second field test too, as well as pulling in another team member to add capacity to the M&E team.

By the time we embarked on the third field test - this time with Street Child in a second country - we had already learnt from the first two, and made further improvements to monitoring and the teacher workshop accordingly. We increased the amount of time for teacher practice during the workshop and added questions on how Chalkboard Guides supported teachers’ national curriculum obligations to find out if there were any obstacles to account for.

Original CBG concept

The 1st set of CBGs we tested

Sample from the latest field test in February 2022

Data-driven development 
Setting the right indicators and having robust means of monitoring them was fundamental to prototyping. We already know that structured lesson guides are a cost-effective way of delivering learning gains, so we honed in on adoptability and fidelity, breaking these down to measure both fidelity to the content, and fidelity to layout on the board. We asked teachers about the most significant changes to their practice after using Chalkboard Guides, and headteachers too for a second opinion. Our choice of indicators was informed by the evidence from cognitive science that drove the design of the guides themselves - we wanted to know if the powerful tweaks that we were delivering through the guides were adoptable (they were).

Secondly, we wanted to develop guides that teachers love, so we gave teachers an opt-out step as well as introducing measures to see how Chalkboard Guides made teachers feel, as well as monitoring how teachers presented during the lesson (confidence, expertise, enthusiasm) to cross-reference their feedback.

Three takeaways from a year of iterative design 

• The ability to make adjustments from field-test to field-test really helped us to iron out kinks in our implementation as well as the guides themselves. It means we’ve been able to have three gos, improving each time, at every one of the seven steps. Through this process we have learnt many lessons to help get implementation right and maximise impact. 
• Investing in the front-end design and implementation by using iterative design improves cost-effectiveness in the long-run. Field-tests keep the costs of correcting errors down, where at scale it would be unwieldy and expensive.
• Iterative design provided a framework so we could put teachers at the heart of the design process right from the start: teachers shaped Chalkboard Guides by providing content, testing lessons, comparing different designs, and much more, right down to choosing the fonts. Listening to teachers is paramount when designing for them.

Of course, it is imperative throughout the iterative design process to keep a laser sharp focus on the ultimate goal. Now that we have a working prototype for Chalkboard Guides, the next step is a longer, wider rollout. Together with Street Child (another member of the GSF community), we will be working to find out what kind of impact Chalkboard Guides can deliver. Will teachers use Chalkboard Guides to deliver foundational literacy and numeracy to the most marginalised children across the world? Will they do this in a timely enough manner? There are still questions to answer, but succeed or fail, we will keep sharing the lessons we learn as we test and (hopefully) scale Chalkboard Guides.

For a discussion about anything we have shared here, please email ee-reh.owo@justicerising.org or reach out on Twitter @EeRehLoves.