Asking complex questions improves creative project scores but hurts multiple-choice exam grades

A recent study published in npj Science of Learning suggests that as students acquire knowledge, their ability to ask complex, subject-specific questions improves, which boosts their performance on creative open-ended projects. However, this same high-level questioning ability tends to negatively impact their scores on standard multiple-choice exams. The findings provide evidence that while deep inquiry aids creative learning, it might conflict with traditional testing methods that require a single correct answer.

Asking questions is a basic human cognitive tool used to identify and fill gaps in our understanding. It helps us reduce uncertainty about the world around us. The researchers conducted the study to understand how acquiring knowledge over time changes the way people form these questions.

While asking questions is a fundamental part of learning, little was known about how the originality and complexity of these questions evolve in a real-world classroom setting. Tuval Raz and Yoed N. Kenett, scientists at the Technion Israel Institute of Technology, designed the research to explore this progression. They also wanted to see whether a student’s natural ability to ask good questions predicts their academic success on different types of assignments.

“In our lab, we research question asking and what makes a certain question more efficient at gathering information. The logical next step was to leave the lab and explore questions ‘in the wild’ and whether being a ‘better’ asker is advantageous for academic outcomes. This is especially pertinent as we usually encourage questions in classrooms, without fully understanding their influence on academic success,” explained Raz, a PhD student and member of The Cognitive Complexity Lab.

Specifically, the researchers looked at the difference between assessments that have one correct answer and those that allow for multiple creative solutions. To test their ideas, the scientists followed 68 undergraduate students enrolled in an introductory psychology course. The study took place over a single spring semester, with data collected during the first week and again during the final week of classes.

This schedule allowed the researchers to track changes as the students learned new course material. During both testing periods, the students completed tasks designed to measure their question-asking abilities. One task measured general inquiry by asking students to generate unusual questions about common objects, like a pencil, sock, pillow, or clock, within a two-minute window.

A second task measured subject-specific inquiry by having students generate questions specifically about psychology and scientific experiments. During the first testing session, the students also completed a scientific creative thinking test. For this assessment, the students read hypothetical scenarios and generated related research questions, hypotheses, and experimental designs.

The researchers scored the generated questions based on three distinct metrics. The first metric was fluency, which simply counted the total number of questions a student produced within the time limit. The second metric was originality, which used artificial intelligence models trained on human ratings to evaluate the creativity and uniqueness of the questions.

The final metric was complexity, which the scientists evaluated using a concept known as Bloom’s taxonomy. This is an educational framework that ranks cognitive tasks from basic factual recall at the lowest level to creating and synthesizing new ideas at the highest level. These levels include remembering facts, understanding ideas, applying information, analyzing connections, evaluating decisions, and creating new work. Using this scale, a basic question asking for a definition would score lower, while a question requiring deep analysis would score higher.

At the end of the semester, the researchers compared the students’ question-asking scores against two different final academic assessments. The first was an open-ended group research project that required students to propose and conduct a small scientific experiment. Open-ended tasks reward what psychologists call divergent thinking, which is the ability to generate multiple different ideas or solutions to a single problem.

The second assessment was a closed-ended multiple-choice final exam covering the course material. Closed-ended tests reward convergent thinking, which is the ability to quickly narrow down information to find the single correct answer.

When analyzing the data, the researchers noticed a shift in how the students asked questions over the semester. As the students learned more about psychology, their subject-specific questions became more numerous, original, and complex. On the other hand, their general question-asking abilities either stayed the same or declined in originality.

The study also revealed specific links between questioning skills and academic performance. Groups of students who generated highly original and complex subject-specific questions at the end of the semester tended to earn higher grades on the open-ended research project. Generating a large quantity of questions was associated with lower project grades, suggesting that quality mattered more than sheer volume.

The researchers noted that the timing of complex questions also played a major role in student success. High complexity at the beginning of the course, before students had mastered foundational concepts, actually predicted lower scores on the group project. The scientists suspect that generating complicated questions too early might overwhelm the students.

“We were especially surprised to see that more complex questions in the beginning of the semester predicted poorer final test performance, whereas complexity at the end of the semester was beneficial,” Raz told PsyPost. “We understand this to mean that before acquiring relevant knowledge, high complexity is a sign of overcomplication. Conversely, complexity at the end of the semester is a sign of material mastery and is thus beneficial.”

When it came to the multiple-choice exam, the results showed an opposite trend. Students who asked more complex and original questions tended to receive lower grades on the closed-ended test. The researchers suggest that the creative, exploring mindset required for complex questioning does not align well with exams that demand rapid and precise factual recall.

This highlights a potential mismatch in modern education. Schools often teach and encourage skills related to dealing with uncertainty and creative inquiry. Yet, they frequently test students using standard exams that reward an entirely different set of rigid memory skills.

“Although question asking is generally viewed as something we encourage, our findings show that it may harm us in the long run,” Raz said. “This is especially true regarding closed-ended standardized tests — which are the norm in academia. Perhaps we should shift to more open-ended grading and tasks.”

While the findings provide evidence that questioning skills shape educational outcomes, there are some limitations to consider. The sample size was relatively small, and the research was restricted to a single psychology course. This means the results might not automatically apply to other academic disciplines or age groups.

Another detail to keep in mind is the difference in how the final assignments were graded. The open-ended project was completed and scored as a group, while the multiple-choice test was an individual metric. This difference in grading structures could potentially influence the comparisons between the two types of assessments.

In the future, the scientists plan to continue their longitudinal tracking of student inquiry skills over time. “We hope to further explore longitudinal research on question asking and are now in the midst of running a manipulation study aimed at enhancing academic outcomes through question asking,” Raz said.

The study, “Knowledge reshapes inquiry by changing question asking ability and impacting academic assessment,” was authored by Tuval Raz and Yoed N. Kenett.