Adapting transformers towards a balanced metagame through retraining and layer-pruning

With the recent strides in machine learning and a growing interest in using Transformers in many different areas, the question has arisen of what can this architecture do for improving the quality of games with procedurally generated characters, such as the Pokemon game. In this context, the overall aim of this thesis is to produce sets of Pokemon (metagames) where the individual Pokemon in a set have more balanced win rates (computed in battles) and more diversity (different Pokemon property values), when using a Transformer model to generate Pokemon. In order to address that aim, this thesis offers two main contributions, involving two proposed methods for improving the Pokemon metagame generated by a Transformer. The first contribution is the approach of pruning layers of a Transformer model, whilst the second contribution is the development of a multiobjective genetic algorithm for selecting Pokemon that optimise the balance and diversity of a Pokemon metagame. The computational results have shown that layer-pruning is paramount in preserving Pokemon diversity over time - every Transformer model produced without using layer pruning was significantly outperformed (regarding diversity) by models generated using pruning. In addition, the combination of layer pruning with a multi-objective genetic algorithm further improved the results, achieving overall the best results in terms of maximising diversity and balanced win rates over time.