Data Driven Impact: How data helped me improve women’s health in Pakistan
Data Science has been making waves in the world for the past few decades - from data based decision making & recommendation systems to transforming how we shop, make investments, or detect diseases.
Personally, I am most intrigued by applications of Data Science & AI in the social impact domain to address the UN Sustainable Goals. Organizations like DataKind, DSSG, Data for Good & Correlaid are a few of many entities working on public & non-profits data to make AI based solutions impacting millions of lives. Some of my favorite applications are:
And the list just goes on…..
Society is at a moment of technological transformation that holds great promise in building data-driven social impact. And with the increasing digital era, we are collecting more data than EVER. Yet, many organizations (especially in Pakistan) struggle to develop evidence-based practices and true learning to use this data to scale their impact.
We at Aurat Raaj work at the intersection of adaptive learning, data patterns, and innovation to address these challenges. We have improved our approach, better understood our user and received recognition & validation across the globe on the basis of our Data Driven Impact.
When we started off in 2017, we relied on very basic surveys and information such as number of workshop attendees, social media views & engagements and website interactions to measure our impact.
But as time passed we collected better data through our chatbot. Let’s have a closer look at how we have been using this data to measure impact and improve our innovation.
Phase 1 - Collecting conversational data, flagging emergency cases & failed conversations:
Through our first iteration of our open-ended chatbot Raaji, the data we collected was the conversations users had with the chatbot & their unique device IDs to differentiate between users. The purpose of not collecting any user specific data was to ensure the safety & anonymity of women & girls.
Along with that, we also collected the data of emergency cases we received, conversations that were taken and closed by admin/volunteers, conversations assigned by admin to a volunteer and conversations revoked for being irrelevant in our database. We also collected the number of failed conversations by the chatbot and how many of them were later resolved by our volunteers and professionals on board.
Through our conversational data, we also visualized topical concentration of questions we received through the chatbot.
This helped us narrow down that:
Our topic concentration should be Menstrual Hygiene Management as most questions asked by our users revolved around this topic.
We needed more human resources to handle failed & unattended conversations.
Most conversations failed due to the user's inability to start a conversation or lead the chatbot in the right context.
Phase 2 - Collecting knowledge-base data to analyse user’s learning gaps & impact of chatbot:
This led us to move forward to our second iteration of the chatbot which was close-ended & would guide the user into the conversation without having to type much. And in 2019, with this new approach, we piloted in 5 non-profit schools in Karachi with an urdu version of Raaji.
Above insights also helped us realize that the data we were collecting was not actionable & easily comprehensible for the public. And for collective action and long-term government buy-in, DATA is essential.
To evaluate how the students interact with the chatbot - their knowledge improvement & learning gaps, we collected pre & post analysis data from the students in the form of quizzes.
These quizzes are conducted within the chatbot before & after the students get information from the chatbot on Menstrual Hygiene.
This approach was tested with school girls from different areas in Karachi.
Dataset information:
Number of girls: 120 girls
Age Group: 11 - 18 years old
Collected from Locations:
Garage School, Neelum Colony
Aghaz School, Limogth
Ibrahim Ali Bhai Govt. Boy & Girls Secondary School, Orangi Town
Partners
Robin Hood Army Pakistan
Santax Pakistan
While all the insights generated from this data pointed out that Menstrual Hygiene education & access of products is essential in Pakistan, here are some of the most alarming answers we received from girls.
Question: Is menstruation a shameful phenomena?
Before our session, only 29.2% of the girls thought that periods were not shameful. Whereas, after the session this changed to 84.4% of girls believing that periods aren’t a bad phenomenon and it is natural to go through this.
Question: What can one use during periods? OR Can you use anything for periods?
Before the session, almost half the girls i.e. 46.7% believed they could use any product for menstrual blood like cloth, rags, cotton, etc.
Through our interventions, we educated them about why they should use a sanitary pad, its benefits and how to use it. After that, approximately 85% of the girls were convinced to use sanitary pads while on periods.
Question: Do you miss school while you’re on your period?
Before our session, only 44.17% of the girls said they didn’t miss school during periods. Whereas, after our interventions it led to 83.19% of the girls into believing that they shouldn’t miss school during periods.
Again data helped us conclude:
Continuous interventions from time to time are important for improved menstrual hygiene of girls. Therefore, Raaji should be deployed in schools & teachers should be trained to use the tool with girls.
There should be a data pipeline to monitor and evaluate a child's learning.
Mothers should also be educated to unlearn so culturally rooted myths about menstrual hygiene as they play a significant role in the lives of young girls.
Future Enhancements: Building Data Dashboard for monitoring & evaluation of Impact created & behaviors changed from unhealthy to healthy:
These insights gave birth to the Ask Raaji campaign we will initiate in 2020 to deploy Raaji in schools across Pakistan.
To monitor and evaluate how girls interact with the chatbot and measure impact at scale, we aim to build a Data Dashboard that will show real time insights about the student’s activities such as topics studied, pre & post analysis, time of usage of chatbot, etc.
This data will also be useful for teachers, schools, government & NGOs to take contextualized action. Here is a sneak peak of our very cool dashboard!
We believe that the stories of girls we tell through data will help identifying the needs of these girls & help create a movement to put an end to misinformation and consequences due to poor reproductive health of women in Pakistan.