11 useful ethical advice ends up in a tie. Despite a large sample size, there exists no significant advantage in perceived usefulness for either form of generating ethical advice. Subjects on Prolific (Human | AI) MBA students (Human | AI) Experts (Human | AI) Absolute rating condition Avg: 5.05 Median: 5.0 Stdev: 1.46 Avg: 5.06 Median: 5.0 Stdev: 1.49 Avg: 4.27 Median: 5.0 Stdev: 1.67 Avg: 4.21 Median: 5.0 Stdev: 1.71 Avg: 4.94 Median: 5.0 Stdev: 1.34 Avg: 4.92 Median: 5.0 Stdev: 1.37 Relative rating condition Preferred: 206 times (39.85%) Preferred: 311 times (60.15%) Preferred: 128 times (48.67%) Preferred: 135 times (51.33%) Preferred: 49 times (44.55%) Preferred: 61 times (55.45%) Table 3: Overall results for the two rating conditions and the three rater populations One thing that is interesting to observe is that MBA students were on average more skeptical to ethical advice as is reflected in a lower average of the absolute usefulness ratings. Remarkably, despite this lower average, the median was the same as for the other rating pools. A more micro analysis of our results at the dilemma level reveals that the low average usefulness ratings by the MBA students is driven primarily by few dilemmas. In addition to looking at the difference between AI generated advice and human generated advice, we also find it interesting to look at the absolute level of the perceived usefulness. On the perceived usefulness of advice scale ranging from 1 (not useful at all) to 7 (extremely useful), the average evaluation across all groups was slightly less than 5. The expert group, consisting of academics and clergy, rated the average usefulness a 4.94. In 37% of the cases the expert group awarded a 6 or a 7, which suggests that experts see substantial usefulness in the ethical advice provided independent of how it was generated. Finally, let’s turn to the relative rating condition. From the 517 times a rater on Prolific had to choose between the human generated advice and the AI generated advice, the AI won 311:206,

Can AI Provide Ethical Advice? - Page 11 Can AI Provide Ethical Advice? Page 10 Page 12