Friend´s Autolarm

Purpose and goal:

The project aimed to use AI to analyze surveys from school students in Sweden and detect early signs of bullying. Data was collected from 30,000 surveys, with 600 annotated responses in categories of victimized/not victimized. The model´s goal was 85% accuracy, 80% precision and recall, and an F1-score of 0.82.

The model was designed to be sensitive and rather produce false positives. Pre-built language models like ChatGPT and Llama2 were compared with the project´s AI to measure effectiveness. Expected results and effects:

We utilized our own survey tool tailored for schools and developed AI to detect abuse and bullying. When compared against an LLM, the latter outperformed the former. Our AI discerned just ´bullying´ or ´no bullying´, while the LLM differentiated several categories, offering deeper insights. This result, driven by our model´s data limitations, was foreseeable. LLMs might outshine in precision. However, with ample data, the custom model built for the project holds potential.

Approach and implementation:

The project used deep learning and LSTM for sentiment analysis of data from Friends and app reviews. Data processing included text cleaning and the creation of numerical representations. Data was trained in different rounds, and after each training round, performance metrics were evaluated. Adjustments were made to improve the model´s performance. Pre-trained language models were tested and showed equivalent performance. Collaboration with Friends ensured the quality of the text analysis