Dverse Social AI - Controlling AI Participation in Group Text Chat

Project description

This project investigates how controlling the participation frequency of an AI agent in a group text chat affects the quality of academic discussions.

A Streamlit-based prototype called the AI Discussion Room was built, in which two participants collaboratively construct a research gap from a set of academic papers alongside a visible AI participant. The AI signals its intent to contribute using a GREEN/BLUE/RED urgency indicator, and a threshold rule determines when it may intervene.

Two conditions were compared across eight sessions: a Conservative threshold (AI responds only at highest urgency or when explicitly called) and a Proactive threshold (AI also responds at lower urgency levels).

Context

Students in higher education frequently struggle to construct research gaps collaboratively without sufficient critical challenge or structured support. Existing AI tools typically function as individual assistants rather than active group participants.

This project explores a less-studied design space: AI as a visible, controllable participant in real-time academic group discussion, grounded in retrieved literature excerpts to reduce hallucination risk.

Results

The Proactive threshold produced substantially higher AI participation (39.2% of turns, 67.1% of words) compared to the Conservative threshold (13.5% and 29.3%). Despite this, perceived discussion quality and AI helpfulness ratings were nearly identical across conditions. The Conservative condition was rated more favourably on AI timing (4.17 vs 3.60 on a 5-point scale).

Human word output dropped by approximately 57% in the Proactive condition, raising concerns about cognitive offloading. The most consistent finding across both conditions was that participants found the AI too agreeable, preferring an agent that challenges rather than validates their reasoning. This suggests that AI response style may matter more than participation frequency as a design variable.

About the project group

This project was carried out as an individual Master of Applied IT research study, combining prototype development with an experimental study design.