AI-Assisted Learning
Understanding the extent of AI chatbot usage amongst primary and middle school children in Pakistan.
Role
Primary & Secondary Research, Findings Analysis, Paper Writing
Team
Noverah Khan, Hira Eiraj, Dr. Suleman Shahid
Tools
Miro, Slack
Timeline
5 Months
Introduction
In Pakistan, while AI chatbots are gaining popularity, there is a lack of research on their use in primary and middle schools, highlighting the need for guidelines on their effective integration. Thus, we seek to understand the extent of AI chatbot usage amongst Pakistani primary and middle school children. Based on our findings, we recommend guidelines on how to prepare children to critically and effectively use AI assistance to their advantage.
User Research
First, we conducted a 5-minute survey with 156 students (grades 2-8) in Lahore, Pakistan, to assess their awareness and usage of chatbots. Next, we conducted in-depth interviews with 30 students (grades 5-8) to explore chatbot usage, safety perceptions, and discussions with parents and educators. These were followed by additional interviews with 10 teachers (grades 5-8) to gauge their perspectives on students’ AI chatbot use. The data was analyzed using thematic analysis (with inductive coding).
Findings
Students:
Students in grades 2-4 showed little to no awareness of AI chatbots and had only used the tool once or twice (mostly for fun/entertainment purposes).
Students in grade 5 and above had more frequent and mindful usage (particularly with regards to academic work), leading us to focus on grade 5 and above for interviews.
Interviews revealed that though grade 5 students interacted frequently and mindfully with the chatbot, their interactions were primarily exploratory in nature.
A notable shift occurred from grades 6 onwards, with students using it regularly for research and project/essay ideas, particularly for subjects like English, Science and Math.
Only one grade 8 student raised concern regarding the accuracy of information provided by chatbots, while the rest considered them safe and reliable.
According to students, their parents saw it as a useful tool, though some set boundaries on its use, particularly for homework and inappropriate content searches, reflecting varying levels of comfort with AI technology.
Teachers:
Teachers expressed mixed opinions on ChatGPT, with some seeing it as a valuable tool for research, while others were concerned about overreliance and its impact on learning.
They emphasized the need for age-appropriate use.
Urdu teachers observed a lack of chatbot usage in their classes, speculating it might be due to its perceived english-centric nature.
English and Math teachers were especially cautious about usage for assignments, fearing plagiarism in essay-writing and dependency on the tool for solving math problems, while Science and Social Studies teachers acknowledged its value in aiding research for their subjects.
Several teachers stressed the importance of teaching students to ask the right questions and not to simply copy-paste responses, ensuring they understand the material.
They highlighted the importance of parental involvement in monitoring students' use of chatbots, to prevent misuse and encourage independent learning.
Recommendations
Students should learn to critically engage with ChatGPT's output, avoiding blind copying to enhance language skills and counter misinformation.
For English, children should be taught to avoid using it entirely, as any reliance on the tool can impair their language development.
For Science and Social Studies, it can aid research and supplement existing knowledge, but students must fact-check answers to prevent misinformation or identify any harmful biases that it might perpetuate.
In Math, students should seek to understand the methodology behind answers generated by ChatGPT, since knowing how to solve a problem is more important than simply getting the right answer.
Educators should encourage students to ask detailed questions as input to get the best responses and avoid irrelevant or inappropriate content as the output.
Students should be discouraged from relying on it for coursework in languages like Urdu, as the responses may not be on par with native speakers’ proficiency in vocabulary and sentence structure.
Schools should introduce a module in their ICT course on effective, responsible ChatGPT use, incorporating assignments that guide students on how to use it critically for schoolwork, and guidelines regarding plagiarism and fact checking. They should also be taught about how LLMs generally work (with training data bias).
Future Work:
We plan to interview parents, conduct assignment-based activities to observe how children use ChatGPT, and expand our research to different schools across Pakistan, including public schools, to refine our recommendations and develop comprehensive guidelines for both parents and teachers based on diverse socioeconomic and cultural insights.
Publication
Noverah Khan, Hira Eiraj Daud, and Suleman Shahid. 2024. ChatGPT In The Classroom: Guidelines for Integrating AI-Assisted Learning into Pakistani Children’s Academic Journey. In The 13th International Conference on Information & Communication Technologies and Development (ICTD 2024), December 09–11, 2024, Nairobi, Kenya. ACM, New York, NY, USA, 6 pages. https://doi.org/10.1145/3700794.3700813