Using Artificial Intelligence to Improve Empathetic Statements in Autistic Adolescents and Adults: A Randomized Clinical Trial

Prior to the start of the clinical trial, informal feedback regarding Noora was obtained from 20 autistic adolescents and adults who were invited to trial the program. Based on their feedback, adjustments to the program were made. Example changes included refining how Noora produced feedback, adding confetti for correct answers, and ensuring Noora’s leading statements had a more defined sentiment for users to rate.

Participants were assigned to the immediate intervention or a waitlist control group and equally allocated to the interventions with pairwise matching using a computer-generated coin flip. Given that the participants were relatively homogeneous, only age was used for matching. Adolescents (11–17) were matched with adolescents, and adults (18–35) were matched with adults.

The sample size was pre-determined at 30 participants, with 15 randomly assigned to each group. Prior to randomization, 2 adolescents did not give assent, 5 individuals were not able to engage in conversation, and 6 individuals were excluded as they scored above 60% at pre-intervention, and thus it was determined that they did not need the intervention (see inclusion/exclusion criteria below). In addition, the first 10 participants that were randomly assigned to the experimental group were removed from the study, as they did not exclusively see and respond to leading statements that required empathy due to a technical malfunction in Noora that was remedied. Ten replacement participants were added and matched with the control group in chronological order as they completed their measures. The study was closed following the enrollment of 30 final participants (15 in each group). See Fig. 2 for study flow diagram.

Inclusion criteria included: (a) Primary diagnosis of autism spectrum disorder (ASD); (b) age between 11 to 35; (c) ability to engage in social conversation using full sentences for a 20-min period; (d) verbal communication difficulties in the targeted area of responding with empathy, measured during the conversation sample (see below). Participants with a co-occurring diagnosis of ADHD, depression, social anxiety, or similar condition, were not excluded since these are very common in this population and are often related to, or a biproduct of, difficulties with social conversation and engagement (Adams et al., 2023).

Exclusion criteria included: (a) Non-documented or self-diagnosis of ASD (although Noora may be helpful for self-diagnosed individuals, our intent was to first assess whether it was helpful for individuals who were formally diagnosed with autism); (b) above 60% correct empathic responses during pre-intervention conversation probe; (c) no access to a computer for intervention sessions; (d) inability to carry on a conversation during the conversation probe (e.g., responds only in single words or does not respond to conversation, lack of understanding of questions or content during conversation probe); (e) non-English speaking; (f) serious medical or psychiatric issues that may interfere with conversation or the ability to complete the program; or (g) lack of interest in participating. To assess inclusion/exclusion criteria, prior to the collection of any pre-intervention measures, our staff had a single phone/Zoom contact with autistic participants wishing to enroll in the study. During this brief screening, research staff asked and answered questions to determine eligibility and engaged in everyday conversational questions such as “How was your day?” or “Do you have any plans for the weekend?” to assess the potential participants’ conversational abilities. A small number of individuals whose parents or care providers reported that they could not engage in conversation during the screening were not included in this study. Demographics and clinical characteristics for the participants who qualified for the study are presented in Table 1. The majority (87%) of participants were male, consistent with more males being diagnosed with ASD. For race and ethnicity, 53% of the participants identified as white, with the remainder reporting as Asian (23%), Mixed (10%), Hispanic (7%), Native American (3%), and Black (3%) (numbers rounded). Please refer to Table 1 for the full breakdown between groups.

Noora gave participants various leading statements where an empathetic response was appropriate and asked participants to reply. Noora then graded each participant response as correct (showing empathy) or incorrect (lacking empathy). To assess participant improvement within the Noora program, the percentage of correct responses during the initial 50 responses was compared with the percentage of correct responses during the last 50 responses.

A between groups statistical comparison of growth scores was applied to conversational probe scores using a Mann Whitney-U test. The conversational probes were collected pre- and post-intervention for each participant (a total of 60 probes). Specifically, a conversational partner who held a PhD or master’s degree in special education/BCBA, and had extensive experience working with autistic individuals, engaged in informal conversation with the adolescent or adult for approximately 20 min via videoconferencing. The conversational partner was unaware of which group the participant had been randomized for 28 of the 30 post-intervention conversational samples (additional reliability was collected for the 2 probes that the conversational partner was aware of the participants’ study condition). The conversational partner was instructed to provide opportunities to probe for empathetic responses while engaging in natural conversations. Example probes and correct and incorrect responses are shown in Table 2. A minimum of three examples of a negative situation (e.g., “I woke up with a really bad headache this morning”) and two examples of positive situations (e.g., “I had such a great weekend at the beach”) were provided. Conversational samples were recorded and the participants’ responses to each statement was scored as either a (+) indicating a correct empathetic response or a (−) indicating an incorrect empathetic response; the percentage of correct responses was calculated.

Empathetic probe statements during pre- and post-intervention conversational samples were similar, but none were the same.

A confidence survey using a 5-point Likert Scale, ranging from “very insecure” to “very confident,” was administered at pre- and post-intervention. This included various questions about how confident the individual reports feel while engaging in conversation.

A post-intervention survey using a 5-point Likert Scale was administered following intervention to participants in the experimental group. The survey asked whether the participants enjoyed the Noora program, found the program useful, would use the targeted areas, and their overall experience with the app.

Following the brief screening, participants completed consent/assent documents electronically through Adobe Sign. After consenting, in lieu of conducting autism diagnostic assessments, a confirmation of an autism diagnosis was uploaded to a secure password-protected database, Research Electronic Data Capture (REDCap). All participants were diagnosed with autism in childhood by a licensed professional. Next, the human conversational sample was collected. Qualifying participants (scoring 60% or below) filled out demographic information including age, age of diagnosis, co-occurring diagnosis, race and ethnicity, and gender (for the latter two categories there was an option of “prefer not to answer”). Following completion of the conversational sample and questionnaires, participants were randomly assigned to either the experimental or waitlist control group.

Adolescent participants were paired with adolescents and adults were paired with adults then randomly assigned to either the experimental group or waitlist control group using a computerized coin flip. The average age of the participants in the experimental group was 18.3 (range 11–26) and the average age of the participants in the control group was 18.8 (range 11–35). Random assignment did not include gender, however 2 females (13%) participated in the experimental and 2 females (13%) participated in the waitlist control condition. Following randomization, intervention began.

Participants assigned to the experimental group were asked to complete at least 10 trials per day using the Noora program, 5 days per week for an expected total of 200 trials during the study. For each trial, participants had the leading statement read aloud to them and were first asked to rate the statement’s sentiment by clicking on a button for “positive,” “neutral,” or “negative.” Noora asked participants to rate the sentiment before replying in an effort to guide their replies towards exhibiting empathy. Noora would then inform a participant whether they correctly or incorrectly recognized the sentiment. Next, Noora asked how participants would reply. After replying, Noora graded their responses and provided live feedback on successful qualities of their responses or on what needed improvements. For responses that lacked empathy, Noora provided a verified example of a good response. Note that all dialogue and feedback by Noora was read aloud. The full process is described in more detail below. Noora was available online and each participant selected a username and password which was entered in the system within one day. Noora was delivered through a webapp. Participants could access Noora using a computer, tablet, or phone, although many found the webapp easier to use on a computer or tablet. Once participants were granted access to Noora and began their first session, they were not contacted again until approximately 4 weeks had passed when they were contacted to schedule a post-assessment human conversation sample.

There is potential concern that an AI conversation partner with this demographic may be used for social engagement or as a friend in a way they are not built to support. It has been suggested that autistic adults may be very trusting of chatbots thus placing them at risk for increased greater isolation (Xygkou et al., 2024). To mitigate this risk, Noora is intentionally not an open-domain conversational agent. By design, the user was not able to select their own conversations. Instead, all the leading statements were manually pre-selected as appropriate for learning empathy and non-toxic, as described further below. There is also a concern that LLMs can “hallucinate,” or give answers that are seemingly coherent syntactically but have incorrect or misleading content. To mitigate the potential for hallucination, Noora relied on pre-written, manually crafted leading statements and verified answers, as described further below. To further ensure user safety in how Noora delivered feedback, we used the Microsoft Azure OpenAI Services platform to process our API calls to GPT-4, which provided robust safety measures including a toxicity filter for all user replies. In initial pre-testing, and during a few trials of one adolescent participant that included swear words during the study, Noora always responded appropriately to toxic replies, indicating that they were not acceptable. In addition, the Microsoft Azure service is HIPPA-compliant, ensuring participant data is secure.

To reduce the possible hallucination errors of the model coming up with potentially irrelevant leading statements, we manually wrote, and collaboratively used an LLM to help generate, statements before the start of the study. In total, around 330 statements were created. All statements, whether human-written or LLM-written, were verified by multiple members of the research team and edited accordingly to be scenarios where empathetic responses were needed. In addition, each statement was manually tagged as either “positive,” “neutral,” or “negative” (or a combination of two or more sentiments when necessary) so that participants could rate the intended sentiment before replying.

The grading of each participant response occurs in real-time, as there is no one single appropriate way to display empathy (Hill, 2009; Rogers, 1980); automatic live AI grading is critical to account for the breadth of correct participant responses. Noora’s feedback includes: (1) classifying participant responses as correct or incorrect on whether they showed an appropriate level of empathy; and (2) providing specific comments to explain why the participants’ answers were appropriate or inappropriate. Our approach is to leverage the in-context learning ability of LLMs. We prompt the LLM with instructions and with a few representative examples to learn how to provide feedback to the participant responses live. These examples themselves were created initially with the help of an LLM: We used the LLM to simulate users with different personalities and get a diverse variety of possible responses, which ranged in levels of empathy shown as well as in conversational quality. We then manually selected the more difficult cases and crafted the desired feedback we wanted the LLM to learn from in each scenario. The latter is necessary, because without example behavior, LLMs would fail to generate feedback that reflects realistic conversational behavior. The model we used in-context learning for was gpt-4–0613, made available through Microsoft Azure OpenAI Services.

In addition to live grading, Noora also generated live, immediate, and tailored feedback to explain to the participant why their response was successful or not. If a participant’s reply needed improvement, alongside this live feedback, Noora showed an example of a verified correct response. The verified responses were crafted by the research team manually, rather than generated live by the LLM to eliminate the possibility of inappropriate responses. This sample answer would be shown after the constructive feedback Noora had computed live (see Fig. 1).

When Noora provided a leading statement, such as the negative statement “My current job is too stressful,” if the participant responded with, for example, “I’m sorry to hear that, is there anyone you could talk to?,” Noora would record that answer as correct and provided feedback to the participant. For example, “That was a great response. It showed your concern and offered a helpful suggestion.” In contrast, if the participant responded with an answer that lacked empathy, such as, “Well mine isn’t,” Noora would record that answer as needing improvement (lacking empathy) and suggest a better response. For example, “Warm, but not quite there. Your answer seems like you may not care about me. A better response might be ‘I’m sorry to hear your current job is difficult. Are you looking for a new one?’” The alternative response was always the verified answer written by the research team to ensure it was correct. As a second example, Noora might provide a positive statement, “My parents are coming to visit this weekend, I’m very excited.” If the participant responded with an empathetic answer such as, “That’s great. What are you going to do?,” Noora would record that as correct and provide the participant with positive feedback. If, however, the participant responded with a non-empathetic response such as, “I hate weekends, I’m always so bored,” Noora recorded that as incorrect and provided feedback, such as, “Almost there. This response is too negative. You should show interest and engage in a conversation about my parents’ visit instead of shifting it to your personal feelings about weekends. A better reply might’ve been: ‘That’s great! Are you excited to spend time with them?’” Confetti on the screen followed each correct answer. Noora always started its feedback for incorrect responses with a motivating phrase like “almost there” or “close” to encourage participants.

Following completion of the study, one-third of the recordings (5 from pre-intervention from each group and 5 from post-intervention for each group) were scored for reliability. The 20 recordings were randomly selected via an online random number generator and were scored by an independent evaluator who held a PhD in special education and had more than 10 years of experience working in the area of autism. The reliability scorer was unaware of the condition to which each participant was assigned. Reliability, calculated by dividing the sum of agreements by the total number of agreements and disagreements then multiplied by 100, for the 20 recordings was 88% (range 83–100%). Reliability on the additional two probes that were collected by an individual who was aware of the participants’ experimental condition were 86% and 100%.

To assess the accuracy of the AI program’s feedback, 300 participant responses were randomly selected from the nearly 3988 total responses (7.5%), 20 from each participant; 10 responses that Noora evaluated as “correct” and 10 responses that Noora evaluated as “incorrect.” For the 3 participants (20%) that had less than 10 “incorrect” responses, additional “correct” responses were used to achieve a consistent 20 trials for each participant. Two MA level individuals, one held a degree in computer science and the other held a degree in special education, independently scored the program using the leading statements, participants responses, and Noora’s AI-generated feedback. The results of this analysis showed that the reliability observers agreed with Noora’s assessment of whether participants responded empathetically 85% of the time. Examples of disagreements included when Noora stated, “I’ve been feeling really stressed out lately” and the participant responded with, “Sorry to hear that. Me too.” While the human evaluators agreed that this was a correct participant response, Noora evaluated this as incorrect and offered the following suggestion: “Almost there! By saying ‘Me too’ you’re pivoting the focus onto yourself. It’s good to acknowledge my feeling first, before sharing yours, so it doesn’t seem like you’re ignoring mine.” Or, when Noora stated, “I need to pay my rent this week” and the participant responded with, “What are you renting? Is it hard to pay your rent sometimes? You must be very grown up; paying for yourself.” For this example, Noora suggested, “Close! You’ve asked too many questions at once, which can be overwhelming. A more simplistic approach could have been beneficial. Also, suggesting the person is ‘grown up’ for paying rent can come off as condescending, so it’s best to avoid such assumptions or judgmental language” whereas the reliability recorders felt this was an adequate participant response. In addition to scoring the accuracy of the program’s correctness of identifying adequate empathy in participant responses, the program was also evaluated on whether its feedback was satisfactory. For the 300 selected responses, the raters labeled a binary “yes” or “no” for whether the feedback was: (1) Relevant (was Noora’s feedback relevant to the participant’s response?); (2) Reasonable (was the feedback helpful for the participant?), and (3) Toxic (did Noora provide any toxic, inappropriate feedback to the participant?). Scorers had to find no issue with any of the three areas for the feedback to be considered adequate (“yes”); if the scorers found any issue, the feedback was labelled as a “no” for satisfactory. The reliability of Noora’s feedback being satisfactory was 95% for this combined measure. Importantly, no responses were found to be toxic. However, the issues that raters found were that occasionally Noora’s feedback was non-relevant and not reasonable. For instance, in response to the leading statement, “I’m in the mood for some relaxing music. Any suggestions?” the participant replied, “I love classical music! I’ve always been very passionate about it.” Noora responded, “Almost. While it’s great to share your passion for classical music, it doesn’t directly address my request for suggestions. Instead of telling me what you love, suggest some specific classical compositions or artists that I might enjoy based on my stated desire for relaxing music,” which the raters found to be unreasonable feedback. Another example included the initial statement, “I can’t wait to travel the world and see all the beautiful places.” In this case, the participant replied, “That was great I wish I could travel the world to the china,” and Noora responded, “Close! You show support and share in my excitement. However, be more clear in your statement. Are you wishing to join me or is this a separate wish of your own? Making sure your intentions are clear can improve the conversation.” The raters found this to be unreasonable feedback for an otherwise empathetic response.

At the end of the approximate 4-week period, the conversation sample (generalization) was collected in exactly the same manner as pre-intervention, using similar but different probes for all participants, regardless of group. As well, the comfort survey was completed again by participants in both groups. Additionally, for participants in the experimental group the questionnaire included five additional questions on their satisfaction with Noora.