Explaining AI Without Code: A User Study on Explainable AI

AI summary

Overview

Research area: Explainable AI (XAI) and human-computer interaction, with a focus on no-code machine learning platforms. Technical level: Intermediate. Scope: the paper presents and evaluates a human-centered explainability module built into DashAI, an open-source no-code ML platform, through a user study with 20 participants drawn from ML novices and experts.

What This Paper Is About

No-code machine learning platforms let people train and deploy models through graphical interfaces, but most of them include little or no explainability, leaving users unable to understand, validate, or trust the predictions they produce. At the same time, most existing XAI methods require programming expertise, so they are out of reach for the non-programmers these platforms are meant to serve. The paper's goal is to close this gap by integrating three complementary explanation techniques directly into a no-code workflow and studying how novices and experts actually use them.

Key Contributions

1. Integration of three complementary XAI methods (Partial Dependence Plots, Permutation Feature Importance, and KernelSHAP) into a no-code ML workflow for tabular classification.
2. A user study with N = 20 that reports a task success ratio, a System Usability Score (SUS), and overall positive satisfaction across both user groups.
3. Insights into differences between novices and experts, along with implications for future human-centered XAI design.
4. A documented set of user test tasks and questionnaires (ESS, TiA, SUS) that other researchers can reuse for evaluating explanations in no-code environments.

Main Findings

• Task success was high across the board: all five explainability tasks achieved at least 80% success (N = 20). Opening the explainability dashboard (T7) and generating and analyzing PFI (T8) both reached 100%; generating and analyzing PDP (T9) reached 90%; generating and analyzing KernelSHAP (T10) reached 80%; comparing PDP, PFI, and KernelSHAP (T11) reached 90%. Most errors occurred in T9–T11, typically during generation, analysis, and comparison of local (KernelSHAP) and global explanations.
• Explanation Satisfaction Scale (ESS) reliability was acceptable: Cronbach's alpha = 0.74. Median scores were 5 (on the scale used) for usefulness and accuracy, and 4 for comprehension, satisfaction, sufficiency of details, completeness, usability, and trust.
• No significant difference between the two explanation workflows: a Mann–Whitney U test comparing the progressive (Scenario A) and combined (Scenario B) conditions returned U = 38, p = 0.38.
• Novices rated explanations more positively than experts: logistic regression yielded a marginally significant effect (p = 0.082). Novices concentrated their responses in Agree/Strongly Agree, particularly on usefulness, accuracy, and trust, and emphasized transparency and reassurance. Experts displayed greater variability and some disagreement, and were more critical of sufficiency of details and completeness.
• Trust in Automation (TiA) reliability was questionable: Cronbach's alpha = 0.60, so the authors advise cautious interpretation. Participants reported moderate to high agreement on predictability and confidence, while familiarity and propensity to trust varied more strongly. Novices tended to trust the system more readily, while experts were more reserved.
• SUS results are not reported in the available content: the contributions list mentions a System Usability Score, but no SUS values appear in the results section provided.

Methodology in Plain English

The researchers built an explainability module inside DashAI, a no-code ML tool whose home screen gives access to Datasets, Experiments, Predictions, Explainers, and Plugins modules. From there, users upload data, configure and train models, and apply trained models to new data. The new module embeds three explanation types directly in that workflow: PDP, which shows how changing one feature affects the predicted probability of the target class; PFI, which ranks features by how much performance drops when each one is randomly permuted; and KernelSHAP, which breaks an individual prediction into feature-level contributions. PDP and PFI give global views of model behavior, while KernelSHAP gives instance-level (local) explanations.

To evaluate the module, the team ran a between-subjects user study with N = 20 participants, split evenly into novices (n = 10) and experts (n = 10). Participants were randomly assigned to one of two conditions: Scenario A, where explanations were generated and analyzed sequentially (PDP, then PFI, then KernelSHAP), and Scenario B, where the three were generated and analyzed together. Each scenario contained 5 novices and 5 experts. The original test had eleven tasks, but only tasks T7–T11 concerned the explainability module; T1–T6 covered prerequisite workflow steps such as loading datasets, training a text classification model on the Twitter Sentiment Analysis dataset, exploring a customer churn dataset, and creating and checking two KNN models. Each session lasted about one hour and included task execution, questionnaires, and a brief semi-structured interview.

Three metrics were used. Task success was the percentage of participants completing each of the five explainability tasks. Satisfaction with explanations was measured with the Explanation Satisfaction Scale (ESS), which covers eight dimensions: comprehension, satisfaction, sufficiency of details, completeness, usability, usefulness, accuracy, and trust. Trust was measured with four TiA subscales: familiarity, predictability, propensity to trust, and confidence in automation. Internal consistency was checked with Cronbach's alpha (values above 0.7 are commonly considered acceptable), group comparisons used non-parametric Mann–Whitney U tests, and novice–expert differences were explored through logistic regression with a median-split transformation of average scores.

Why This Matters

Research impact. Most prior XAI evaluations look at single explanation methods in isolation, and rigorous comparisons between novices and experts remain scarce, especially in no-code or low-code settings. This study provides empirical evidence for that gap and frames a central design tension: explanations that build trust and transparency for beginners are not the same as the depth experts need to diagnose errors and validate robustness.

Real-world applications.

• Healthcare, finance, and public policy, where the paper notes that automated decisions carry severe consequences when errors or biases occur.
• Business analysts and domain experts using no-code platforms who need to justify model-driven decisions to stakeholders.
• Educators teaching ML concepts to non-programmers who need interpretable, hands-on examples.
• Practitioners in industrial inspection and similar human-centered XAI settings, a need the related work associates with the XEdgeAI framework.

Industry relevance. Commercial and open-source platforms such as Google Cloud AutoML, H2O Driverless AI, Orange3, KNIME, and RapidMiner are widely used by business analysts, domain experts, and educators, yet their explainability is often limited to simple global feature importance or static plots. This work shows a concrete way to embed interactive, multi-method explanations into a no-code workflow, which is directly relevant to product teams building such tools.

Future Directions

• Extending the approach to explanations for large language models, focusing on reasoning processes and interactive, context-aware interpretability.
• Developing adaptive explanation strategies that adjust depth and interpretability to a user's expertise and task context, addressing the novice–expert tension the study identifies.
• Improving the measurement of trust in this setting, since the TiA subscales yielded alpha = 0.60 and the authors caution that those results need careful interpretation.
• Investigating why experts rated sufficiency and completeness lower and why most errors clustered in the KernelSHAP and comparison tasks (T9–T11), which currently have the lowest success rates (90%, 80%, and 90%).

Target Audience

This paper benefits XAI and HCI researchers studying human-centered explanation design, developers and product managers building no-code or low-code ML platforms, educators introducing interpretability to non-programmers, and domain experts in healthcare, finance, and public policy who use automated decision tools and need to understand when to trust them.

Authors’ abstract

The increasing use of Machine Learning (ML) in sensitive domains such as healthcare, finance, and public policy has raised concerns about the transparency of automated decisions. Explainable AI (XAI) addresses this by clarifying how models generate predictions, yet most methods demand technical expertise, limiting their value for novices. This gap is especially critical in no-code ML platforms, which seek to democratize AI but rarely include explainability. We present a human-centered XAI module in DashAI, an open-source no-code ML platform. The module integrates three complementary techniques, which are Partial Dependence Plots (PDP), Permutation Feature Importance (PFI), and KernelSHAP, into DashAI's workflow for tabular classification. A user study (N = 20; ML novices and experts) evaluated usability and the impact of explanations. Results show: (i) high task success ($\geq80\%$) across all explainability tasks; (ii) novices rated explanations as useful, accurate, and trustworthy on the Explanation Satisfaction Scale (ESS, Cronbach's $α$ = 0.74, a measure of internal consistency), while experts were more critical of sufficiency and completeness; and (iii) explanations improved perceived predictability and confidence on the Trust in Automation scale (TiA, $α$ = 0.60), with novices showing higher trust than experts. These findings highlight a central challenge for XAI in no-code ML, making explanations both accessible to novices and sufficiently detailed for experts.