Artly: Exploring Digital Artists' Perceptions of AI-Generated Feedback

AI summary

Overview

Research area: Human-Computer Interaction (HCI), specifically creativity support tools, human-AI co-creation, and generative AI in digital art practice. The paper is published at the 14th Nordic Conference on Human-Computer Interaction (NordiCHI '26) and categorized under Human-centered computing (interactive systems and tools; empirical studies in HCI), Computing methodologies (artificial intelligence), and Applied computing (Digital Art).

Technical level: Intermediate. The system architecture and study design are described clearly, but interpreting the findings requires some familiarity with HCI study methodology (Likert-scale self-reports, effect sizes such as Cliff's Delta and Epsilon-squared, and thematic analysis).

Scope in one sentence: The paper presents Artly, a prototype tool that critiques digital artists' own in-progress illustrations using personalizable AI feedback grounded in human-authored learning resources, and reports a between-subjects study with 38 artists comparing a version with and without AI reference-image generation.

What This Paper Is About

Most generative AI tools for creatives are built around one-shot, prompt-based generation that optimizes for efficiency and output rather than for the artist's growth, personal style, or self-expression. The authors argue that a less studied alternative is to have AI critique the artist's own work instead of producing artwork for them, letting artists retain the low-level creation work. The paper asks how digital artists actually perceive such personalizable AI feedback, and separately whether adding a feature that generates reference variations of the user's own illustration helps or distracts.

Key Contributions

1. Artly, a working generative AI tool for art critique rather than art production. The system combines personalizable, style-aware AI feedback on users' own illustrations with human-authored educational material, deliberately avoiding direct intervention in the act of creation.

2. A style-tag interaction mechanism for personalization without prompt writing. Rather than requiring artists to articulate their style in text, Artly extracts suggested style tags from reference images (uploaded, searched from stock libraries, or AI-generated) that users can edit, and these tags steer the subsequent AI feedback.

3. A three-level progressive disclosure design for AI feedback. The feedback is structured as a Feedback Summary, Feedback Details, and Examples for Improvement, following the "overview first, zoom and filter, then details-on-demand" principle.

4. An exploratory between-subjects study (N=38) isolating the effect of image generation. Two system variants (Weak Mode without, Strong Mode with reference generation) were compared with predominantly young but experienced digital artists, examining both self-reports and interaction logs.

Main Findings

• AI reframed from threat to support: In the thematic analysis of open-ended responses, 20 participants described how Artly prompted them to reconsider their attitude toward AI, for example describing "a new picture towards AI where it supports rather than replaces artists" (S-R-8). Eleven participants stated that tools like Artly do not address their fundamental concerns, including copyright infringement, replacement of artists, and, less frequently mentioned, environmental impact. Three participants discussed both sides. Of those reporting a positive shift, three said they already had an abstract idea of how AI could benefit artists and that Artly made it concrete.

• Helpfulness driven by low barriers to critique: 25 participants explicitly commented on the helpfulness of the AI feedback. Participants valued that feedback is readily available and perceived as non-judgmental ("I felt like I was learning how to improve my art rather than feel belittled for trying to learn how to improve," W-1). The combination with educational resources was explicitly praised by three participants, who found it more tailored than searching YouTube tutorials or asking other artists online.

• Main value was new ideas and awareness of mistakes: Participants reported gaining new directions for their work and a fresh perspective on flaws they had stopped noticing. They also reported feeling more creative, broadened artistic perspective, and sharpened critical assessment skills. Despite the broad range of feedback categories, responses did not suggest overload; participants described selectively focusing on categories aligned with their current needs.

• The most proficient participants saw little value: Advanced and especially Professional-level participants reported that the AI feedback lacked sufficient specificity and depth for their level of expertise. The self-report data by expertise level shows this pattern: for "Feeling Creative," the median was 4 for most expertise levels but 2 for Professionals (Epsilon-squared = 0.20, large effect); for "Discomfort," medians ranged from 2.5 for Intermediate to 4 for Professionals (Epsilon-squared = 0.21, large effect); for "Attitude Change," the median was 4 for most groups but 3 for Advanced (Epsilon-squared = 0.14, large effect). Effects were small for Agency (0.02), Control (0.05), and New Ideas (0.05), and medium for Style Adaptation (0.07), Self-Improvement (0.13), and Non-AI Features More Helpful (0.08).

• Reference generation had less impact than anticipated and shifted attention: As reported in the abstract, participants who used the image generation feature interacted slightly less with the AI feedback. They reported feeling more creative after using Artly than participants in the restricted mode, while reporting slightly lower scores on new ideas for their work. The authors had expected the feature to help users explore directions and communicate them more precisely via extracted tags, but also anticipated possible design fixation and impaired agency.

• Not all participants saw benefit: One participant stated that "I still feel AI will only funnel my choice into one direction and not expand my horizon that much" (S-NR-8).

Methodology in Plain English

The researchers built a web application with three parts: a Flutter frontend, a FastAPI backend that orchestrates prompts, and a PostgreSQL database that logs interaction data. GPT-4o mini generates the feedback, extracts suggested style tags from newly added references, and derives stock-image search queries from the user's artwork; Flux.1 Kontext [pro] generates reference variations. Stock image search uses Unsplash, Shutterstock, and Pexels.

The workflow has four modules: upload an illustration, gather references in an inspiration board, define a personal art style through style tags, and get feedback (with a fifth branch for educational materials and a color palette generator). The AI feedback is presented at three depths. The summary table covers five default categories (anatomy, perspective, composition, rendering, coloring) with additional selectable categories (atmosphere, values, textures, practical), plus adjustable expertise level and a creative-versus-technical slider. The details level breaks each category into "Creative," "Works Well," "Areas to Improve," and "Art Theory & Techniques." The deepest level turns general advice into concrete step-by-step actions referencing specific brushes, and links to human-authored resources created by the first author, who is described as a professional digital artist with over 250,000 social media followers who teaches digital art. The material includes video tutorials on art fundamentals, practical guidelines, illustration process videos, and a brush set.

The study was a between-subjects design assigning participants to Weak Mode (no image generation; references from uploads and automatic stock-image search) or Strong Mode (also able to generate watermarked variations of their own illustration, varying pose and composition and taking existing style tags into account, with no control beyond requesting a new generation or editing tags). Participants were recruited through the first author's personal online network; 41 were recruited and three removed because no interaction logs were recorded, leaving 38 (18 in Weak Mode, 20 in Strong Mode). The sample was predominantly young (M = 22.0, SD = 4.62 years) and experienced, with 78.9% reporting three or more years of experience; most identified as Intermediate (52.6%) or Advanced (23.7%), with fewer Beginners (15.8%) or Professionals (7.9%). 63.2% had prior exposure to AI tools, but only 7.9% had used AI tools specifically designed for artistic purposes, and 57.9% were concerned or very concerned about generative AI's impact on the art industry.

Sessions ran online for about 30 minutes, with a tutorial video, a demographics questionnaire, the task using their own in-progress illustration, and a post-task questionnaire with nine five-point Likert items plus three open-ended questions. The study was approved by the Ethics Committee of the Faculty of Mathematics, Computer Science and Statistics at LMU Munich.

Because several participants received feedback on multiple illustrations, interaction data was normalized by the number of uploaded illustrations per participant. The authors refrained from inferential statistics given the small sample and exploratory aim, reporting medians and interquartile ranges, Cliff's Delta for two-group comparisons and Epsilon-squared for comparisons across more than two groups, with effect size interpretations following Field (2013). Qualitative responses were analyzed with thematic analysis: open coding produced a codebook of 31 codes, a second author independently applied it, inter-rater reliability on overlapping segments yielded a Cohen's Kappa of 0.575 (described as moderate to substantial agreement per Landis and Koch, 1977), the coders reached full consensus after discussing discrepancies, and the codes were grouped into three themes.

Why This Matters

Impact on research. The paper addresses a gap the authors identify: AI-generated feedback has been explored in UI design, CAD, visual design, writing, and programming, but not extensively in art-making, even though critique is a standard practice in art education and professional settings. It also contributes a rare direct manipulation of a specific generative AI capability (reference image generation) inside a feedback tool, showing that the feature diverted attention from the feedback rather than supporting it as the designers expected.

Real-world applications:

• Art education and self-directed learning: Students who lack access to instructors who understand art theory could receive structured, category-level critique plus linked tutorials and brush references, with the caveat that experts found the feedback insufficiently specific.

• Professional practice and peer critique: Artists could use a style-tag-grounded critique loop to get a "fresh look" at work they have stared at for hours, without exposing unfinished work to public judgment.

• Creativity support tool design: The style-tag mechanism offers a concrete alternative to text prompting for communicating personal style, which could be reused in design, music, or writing tools where users struggle to verbalize their intent.

• Color and palette exploration: The non-AI color palette generator (analogous, complementary, monochromatic) was one of the features participants could use independently from AI feedback.

Industry relevance. The findings matter for companies building generative AI creative tools such as Midjourney, Adobe Firefly, or Leonardo.Ai, which the authors note center on highly automated prompt-based workflows. The study suggests a commercially relevant product direction — AI that critiques rather than replaces, which speaks directly to the 57.9% of participants concerned or very concerned about generative AI's impact on the art industry. It also indicates a design constraint: adding generation features to a feedback tool can pull users away from the feedback, and advanced users need substantially more specific and deeper critique than beginners.

Future Directions

1. Improve feedback specificity and depth for advanced and professional artists. The most proficient participants found the feedback too shallow for their expertise, and the authors identify this as a key limitation with room for improvement.

2. Rethink how (or whether) image generation belongs in a feedback tool. The reference-generation feature had less impact than anticipated and appeared to divert attention away from the AI feedback; understanding why users interacted slightly less with the feedback when generation was available remains open.

3. Address the fundamental concerns of the participants who remained unconvinced. Eleven participants said the tool did not address their concerns about copyright infringement, replacement of artists, and environmental impact, suggesting these issues are outside what a feedback-oriented tool can resolve.

4. Test whether AI feedback can do more than feel helpful. The related work the authors cite found no positive effect of generative AI on divergent thinking despite participants perceiving it as helpful for inspiration, leaving open whether designed interfaces can increase actual creativity rather than only perceived creativity. The paper also notes prior work using LLMs to generate feedback hard to distinguish from human-written feedback, raising questions about validating feedback quality at expert level.

Target Audience

This paper is most useful for HCI researchers working on creativity support tools and human-AI co-creation; designers and product teams building generative AI features for creative software; art educators and researchers interested in AI-assisted critique and learning resources; and artists and art students curious about a generative AI application that explicitly avoids producing artwork in their place. Readers studying human perceptions of AI in creative professions — including trust, agency, and the beginner-versus-expert divide — will also find the expertise-level breakdown valuable.

Authors’ abstract

Recent developments in generative AI have lowered barriers to image generation, but existing tools mostly optimize for efficiency, producing generic results and offering little support for artistic growth. We present Artly, an AI system that combines personalizable AI feedback with human-authored learning resources. In a between-subjects study with artists, we compared a mode without image generation features against one that allowed to generate variations of users' illustrations. Artly was perceived as helpful for learning and self-improvement, with the exception of the most proficient participants. Participants who used the image generation feature interacted slightly less with the AI feedback. They reported feeling more creative after using Artly than participants using the restricted mode, while reporting slightly lower scores on new ideas for their work. Overall, our findings underline the potential of our feedback approach for supporting artistic growth in a manner that is well received by artists.