SuperLim 2.0

Purpose and goal: ** Denna text är maskinöversatt **

The goal of this project was to build on SuperLim by contributing both with training data for each test, a reference implementation with baseline results for a number of standard models, as well as developing and providing a standardized web-based test environment for comparison between models and publication of results. All these objective have been met and resulted in a complete evaluation framework for Swedish language models, which will contribute to a continued strong development of Swedish language technology and Swedish language models.

Expected results and effects: ** Denna text är maskinöversatt **

SuperLim will contribute to the development of Swedish language models by providing a standardized framework for comparing different models with respect to a number of different parameters and test types. We expect that SuperLim will give users in the public and private sectors and academia a better opportunity to make more informed and accurate judgments for the selection of language models for their concrete needs.

The leaderboard will form a gathering point for the publication of new Swedish models, which has been missing in Swedish NLP. Approach and implementation: ** Denna text är maskinöversatt **

SuperLim has taken its starting point from English frameworks such as GLUE and SuperGLUE, but has further developed tests and leaderboards according to Swedish needs. This has accelerated the project work and made it possible for the project to deliver a comprehensive test framework. We have placed more emphasis on transparency and reproducibility than the English versions, and the tests can be used for word-based models as well as both encoders and decoders. The sorting functions we added to the leaderboard help make the selection easier.