Does the Grammatical Structure of Prompts Influence the Responses of Generative Artificial Intelligence? An Exploratory Analysis in Spanish

Abstract

Generative Artificial Intelligence (AI) has transformed personal and professional domains by enabling creative content generation and problem-solving. However, the influence of users’ grammatical abilities on AI-generated responses remains unclear. This exploratory study examines how language and grammar abilities in Spanish affect the quality of responses from ChatGPT (version 3.5). Despite the robust performance of Large Language Models (LLMs) in various tasks, they face challenges with grammatical moods specific to non-English languages, such as the subjunctive in Spanish. Higher education students were chosen as participants due to their familiarity with AI and its potential use in learning. The study assessed ChatGPT’s ability to process instructions in Chilean Spanish, analyzing how linguistic complexity, grammatical variations, and informal language impacted output quality. The results indicate that varied verbal moods and complex sentence structures significantly influence prompt evaluation, response quality, and response length. Based on these findings, a framework is proposed to guide higher education communities in promoting digital literacy and integrating AI into teaching and learning.