Efficient, finance-specific Large Language Models (LLMs) for portfolio construction

The ever-increasing prominence of algorithmic trading in quantitative finance has necessitated the need for reliable and actionable AI-aided domain knowledge from vast streams of data with multiple modalities. Of particular interest is generative AI, owing to its ability to distil insights from non-numerical sources such as news articles, earnings calls, financial reports, and other textual inputs.

In this context, sentiment analysis from text promises to bridge the gap between market movements caused by geopolitical and socioeconomic events and human actions in quantitative trading. The sentiment contained in online textual sources can drive market movements; such information harbours intrinsic advantages and provides a competitive edge to those equipped with the tools to harness it.

Thus, in my research, we aim to employ tensor algebra, machine learning and natural language processing (NLP) techniques to implement finance-specific Large Language Models (LLMs) for sentiment analysis, ultimately utilising them as recommender systems for trading. Despite its significant potential, this area remains largely unexplored, as the transformer architecture, the building block of any LLM, was only discovered in 2017, while the number of finance-specific LLMs is extremely limited due to the unique vocabulary present in the financial domain.

Given the sensitive and fast-changing nature of the finance sector, our proposed LLMs must be both compressed/efficient and explainable while simultaneously achieving high accuracy. These goals set my project apart from any other research conducted on this topic.

In terms of enhancing model efficiency, this is crucial since the markets are constantly evolving, and the inference time of models needs to be minimised. Moreover, as we aim to make these models accessible to the general public, compressing them is vital to enable operation on edge devices without the need for the vast computational resources typically required by LLMs, which can range from tens of billions to trillions of parameters.

To this end, we will develop various tensor decomposition techniques, as well as utilise pruning and knowledge distillation.

Furthermore, as we aim to apply our LLMs in the finance domain, it is essential to ensure they are explainable and prevent them from "hallucinating" - that is, generating responses that are factually incorrect, nonsensical, or disconnected from input prompts. To achieve this, we will employ various "robustification" techniques such as adversarial training, reinforcement learning and instruction tuning to significantly enhance the model's reliability and accuracy, while aligning its responses with human preferences.

Additionally, by incorporating domain knowledge, we aim to introduce a probabilistic framework to establish theoretical and practical performance bounds for our approach to sentiment analysis, helping us understand how our approach reduces the likelihood of making incorrect decisions. This promises enhanced interpretation and prediction, mimicking how a human analyst might analyse sentiment in financial texts, all executed in a fraction of the time needed by humans for each article, as LLMs are capable of processing hundreds to thousands of words per second.