Video walkthrough

ChatGPT’s feedback credibility and students’ perceived learning: the role of agent-related social perceptions

In depth 12:54 CC AI

paperi.ai
0:00 / 0:00

Raúl Pérez‐López, Rubén González-Vallejo, Laura Lucia‐Palacios, Iguácel Melero‐Polo, Andreea Trifu

ChatGPT’s feedback may feel more credible when the system seems competent and empathetic, but that credibility does not matter equally to every student. This study traces how social impressions become learning perceptions.

Abstract

This paper examines whether students’ agent-related social perceptions of ChatGPT are associated with the perceived credibility of its feedback in academic tasks and whether such credibility, in turn, is related to students’ perceived learning. It also investigates whether the relationship between feedback credibility and perceived learning varies according to students’ sensitivity to the quality of their academic work and their sensitivity to grade-based rewards. To address this, we develop a theoretical model grounded in social perception theory and prior research on students’ experiences with AI-based tools. The model is tested using Structural Equation Modelling (SEM) on data collected from 230 students who completed an academic writing task. The findings reveal that students’ agent-related social perceptions—particularly perceived competence and human-like empathy— are positively connected to the credibility students attribute to its feedback. In turn, perceived feedback credibility is interestingly related to students’ perceived learning in the context of academic writing. Furthermore, the positive link between credibility and perceived learning is weaker among students with higher gradebased reward sensitivity. This research contributes to the educational technology literature by advancing understanding of how students evaluate the credibility of ChatGPT-generated feedback and how such evaluations are associated with learning perceptions. It also shows that agent-related social perceptions play a significant role in students’ credibility judgements. Finally, the findings offer practical implications for educators and students seeking to integrate ChatGPT effectively into academic activities.

Transcript

ChatGPT’s feedback may feel more credible when the system seems competent and empathetic—but that credibility does not matter equally to every student. This study traces how social impressions become learning perceptions. The study asks whether students’ agent-related social perceptions of ChatGPT are associated with the perceived credibility of its feedback, and whether that credibility is related to students’ perceived learning.

It also asks whether the relationship between feedback credibility and perceived learning varies with students’ sensitivity to the quality of their academic work and to grade-based rewards. The model was tested using Structural Equation Modelling on data from 230 students who completed an academic writing task.

The findings connect perceived competence and human-like empathy with feedback credibility, then connect credibility with perceived learning; the credibility-learning link is weaker among students with higher grade-based reward sensitivity. Generative Artificial Intelligence technologies have been integrated into education and have been described as pivotal for enhancing academic skills and competences.

The examples named here include language proficiency, critical thinking, analytical skills, and mathematical problem-solving skills. ChatGPT is one of the most prominent GenAI tools used by students in Western societies. Even so, adopting and implementing GenAI tools and ChatGPT in education remains under debate, with academics and practitioners sharing doubts, worries, and opinions.

A central issue is the credibility of ChatGPT’s feedback, because students increasingly rely on it, often without scrutinizing its underlying information sources and data reliability. The current literature suggests that students assess ChatGPT’s credibility through the transparency, accuracy, explainability, and human-likeness of the information it provides.

Previous evidence indicates that human resemblance reduces students’ skepticism and precautions when they address tasks with ChatGPT. What remains limited is research on which human-like features and social perceptions are associated with feedback credibility, and how that association occurs.

That question matters for students and educators integrating AI into daily academic life. Earlier literature found that ChatGPT’s feedback credibility was positively related to students’ attitude and experience, but its consequences for perceived learning had not yet been addressed.

The study therefore asks whether feedback credibility may be associated with students’ perceived learning, and whether that association varies with students’ personal characteristics. Students highly sensitive to grade-based rewards or to the quality of their academic work had shown different degrees of ChatGPT adoption, raising the question of how important credibility is for students with different sensitivities.

The study has three objectives: to examine whether students’ agent-related social perceptions of ChatGPT are associated with perceived feedback credibility, whether credibility matters for perceived learning, and whether that connection varies with sensitivity to grade-based rewards and academic quality.

The framework is based on social perception theory and previous research on students’ experience with AI tools. It is tested using Structural Equation Modelling with collected data. The conceptual model proposes that perceived warmth, perceived competence, and human-like empathy are associated with students’ perceptions of ChatGPT’s feedback credibility.

It also proposes that the positive relationship between credibility and learning perceptions may vary according to sensitivity toward grade-based rewards and sensitivity to quality in assignments. The figure’s arrows show that warmth and competence connect to credibility both directly and through human-like empathy.

On the right, the credibility-to-learning link is marked as moderated by sensitivity to quality and sensitivity to grade-based rewards, while class group, gender, occupation, and academic year enter as controls. Social perception theory suggests that individuals evaluate others’ behaviors based on perceived warmth and competence.

Perceived warmth refers to apparent friendliness, trustworthiness, and helpfulness. In educational contexts, warmth includes friendliness, trustworthiness, helpfulness, care, and responsiveness, while competence refers to perceived expertise, effectiveness, and the ability to provide accurate, creative, and pedagogically sound feedback.

Students’ perceptions of ChatGPT warmth can affect feedback directly and indirectly through human-like empathy. Students could perceive greater empathy when ChatGPT is friendly, adapts to their emotions, develops a supportive role, and gives appropriate feedback to their prompts.

That greater support could be related to perceptions of credibility. The credibility of ChatGPT’s personalized and constructive responses can be associated with students’ perceived learning by improving perceived understanding and skills, particularly in writing and argumentation.

ChatGPT’s capacity to provide immediate feedback makes the learning experience more dynamic and supports autonomous learning. The cited research reports improvements in grammar, vocabulary, accuracy, and coherence in student writing through personalized feedback.

The relationship between ChatGPT’s feedback and students’ perceptions of learning may be affected by individual characteristics, specifically sensitivity to academic quality and sensitivity to grade-based rewards. Sensitivity to academic quality refers to how demanding a student is when assessing performance in a task, while sensitivity to grade-based rewards is the degree to which students are preoccupied with grades as rewards for performance.

The fifth hypothesis predicts that sensitivity to quality will moderate the positive association between ChatGPT feedback credibility and students’ perceived learning. The sixth hypothesis predicts that sensitivity to grade-based rewards will moderate the positive association between ChatGPT feedback credibility and students’ perceived learning.

Students wrote an essay of approximately 180 to 220 words about factors influencing consumer behavior in the purchase of a new smart watch. The activity was completed in pairs, and students had 40 minutes to complete their essays.

The instructions required an introduction, a body of two to three paragraphs, and a conclusion, along with appropriate arguments, understandable sentences, quotations and a reference list, and attention to grammar and spelling. A survey examined students’ perceptions of ChatGPT’s feedback credibility, their agent-related social perceptions of ChatGPT, and their perceived learning.

The questionnaire was completed in both English and Spanish. The measured variables included perceived warmth, perceived competence, human-like empathy, credibility, perceived learning, sensitivity to grade-based rewards, and sensitivity to quality. All variables were measured using seven-point Likert scales.

The survey involved students who participated in the activity, with 327 students enrolled in the seven courses and 241 completing the questionnaire in two waves, in October 2024 and October 2025. After incomplete responses were eliminated, the total valid sample was 230 students, with a participation rate of 73.3 percent.

Of the valid sample, 145 students belonged to the 2024 data collection and 85 to the 2025 data collection. Structural Equation Modelling was applied to test the proposed hypotheses. An exploratory factor analysis initially yielded five constructs, with perceived competence and feedback credibility forming a macro-factor and sensitivity to grade-based rewards and sensitivity to quality falling into another common factor.

Because that dimensionality contradicted literature treating the concepts as different, a parallel analysis fixed factor extraction to seven theoretical constructs. The analysis revealed seven dimensions corresponding to the seven variables, except that item STR2 of sensitivity to grade-based rewards continued loading on sensitivity to quality.

Perceived warmth and feedback credibility correlated positively, but the correlation was not significant, so H1a was not supported; their association was fully mediated by human-like empathy, supporting H1b. Perceived competence had direct and indirect positive and significant links to feedback credibility, supporting H2a and H2b, while human-like empathy was positively related to feedback credibility, supporting H3.

ChatGPT’s feedback credibility was positively associated with perceived learning, so H4 was supported. Sensitivity to quality did not significantly moderate the credibility-learning association, so H5 was not supported; sensitivity to grade-based rewards did moderate it with a significant negative association, supporting H6.

The credibility-learning relationship was strongest at low grade-based reward sensitivity, intermediate at the mean level, and weakest at high sensitivity, with coefficients of zero point seven three three, zero point six zero seven, and zero point four eight three. In the plot, all three lines meet near the center, then separate toward the right side of the feedback-credibility axis.

The red low-sensitivity line rises most sharply, the blue mean-sensitivity line lies between the others, and the green high-sensitivity line rises least sharply, making the moderation visible in the differing slopes. Feedback credibility was positively linked to perceived learning.

When students viewed feedback as reliable and trustworthy, they were more likely to accept it and integrate it into academic activity. Credibility reduced uncertainty about the accuracy of ChatGPT’s information and encouraged students’ willingness to rely on its feedback.

Unreliable feedback could be ignored or processed superficially, making credibility a critical filter for whether feedback became perceived learning. The association between feedback credibility and perceived learning was less important among students more sensitive to grade-based rewards, supporting the rationale behind the Behavioral Approach Model.

One possible explanation is that reward-sensitive students are more critical of ChatGPT’s feedback, more confident in their own knowledge, or more likely to consult additional sources. This remains only a possible explanation, and more research should examine students’ processes during the written task and the additional information sources they used.

Educators and students should critically evaluate the credibility of ChatGPT’s feedback and other GenAI tools. GenAI may provide incorrect, inaccurate, and sketchy recommendations that can be related to students’ learning perceptions. Educators should recognize GenAI’s limitations when assessing academic writing, focusing on students’ level of detail and giving more importance to specific examples because GenAI tools often provide generic answers.

Students should understand these limitations and use GenAI feedback to augment their own capabilities. The study’s activity was conducted in Marketing courses, so results may differ across disciplines, educational levels, cultural contexts, or students’ prior experience with AI.

The study focused on a written essay, so results may differ for other activities or output formats such as oral presentations or visual materials. The teaching experience was based on ChatGPT, so the findings concern that AI agent, even though ChatGPT is widely used.

Students who saw ChatGPT as competent and empathetic tended to find its feedback more credible, and credibility was linked to perceived learning. But that link was weaker among students especially sensitive to grades.

A derivative work by Paperi · AI-generated script, voice and captions · pages and figures unaltered

Made with Paperi.

Drop in a research PDF — get a narrated video walkthrough like this one, with highlights that follow the narration. Free to start.

Try it with your paper →

More in Computer Science

Methodological issues of the electronic health records’ use in the context of epidemiological investigations, in light of missing data: a review of the recent literature 3:12

Methodological issues of the electronic health records’ use in the context of epidemiological investigations, in light of missing data: a review of the recent literature

Medical records can help prevent disease and improve treatment, even when parts of a patient’s story are missing. But the way those blank spaces are handled can quietly change the conclusion doctors and researchers reach.

Hybrid Threshold Denoising Framework Using Singular Value Decomposition for Side-Channel Analysis Preprocessing 3:54

Hybrid Threshold Denoising Framework Using Singular Value Decomposition for Side-Channel Analysis Preprocessing

A device can reveal its encryption key without saying a word. Tiny electrical traces carry clues about the secret, but noise can bury them before anyone can read them.

Adversarial attacks on graph-level embedding methods: a case study 4:21

Adversarial attacks on graph-level embedding methods: a case study

An artificial-intelligence system can make sense of a complicated network, yet an attacker may not need to touch the cases it must judge. Altering its training examples can be enough to make the system fail.

All 21 papers in Computer Science →