Skip to main content
Kent Academic Repository

On the power of special-purpose GPT models to create and evaluate new poetry in old styles

Sawicki, Piotr, Grzes, Marek, Goes, Fabricio, Brown, Dan, Peeperkorn, Max, Aisha, Khatun, Simona, Paraskevopoulou (2023) On the power of special-purpose GPT models to create and evaluate new poetry in old styles. In: International Conference on Computational Creativity (ICCC). . (In press) (KAR id:101234)


This study investigates the possibility of using GPT-3 models to generate high-quality poems in a specific author's style, through fine-tuning on datasets of poems accompanied by their metadata and automatically generated summaries. Our experiments show that a dataset of only 300 poems is sufficient to generate new poems in the style of a specific author. The evaluation was done through GPT-3 models fine-tuned for binary classification of GPT-3 outputs against the works of the original author. To establish the accuracy of GPT-3-based binary classifiers, we first tested them on a variety of texts and a range of classes, and found that their predictive accuracy is 99% on average. Using this method for poetry evaluation showed that the GPT-3 generated poems were indistinguishable from the original works of Walt Whitman and Rudyard Kipling in an average of 30% and 21% of the cases, respectively. This suggests that GPT-3 can be a useful tool in assisting authors, while further research is needed to turn it into an independent creator. Additionally, the workflow used in this study can be applied to other types of text and provides a way of using GPT-3 models for generating new content from user-provided summaries, when prompt engineering alone is insufficient.

Item Type: Conference or workshop item (Paper)
Subjects: Q Science > QA Mathematics (inc Computing science) > QA 76 Software, computer programming, > QA76.87 Neural computers, neural networks
Divisions: Divisions > Division of Computing, Engineering and Mathematical Sciences > School of Computing
Depositing User: Marek Grzes
Date Deposited: 11 May 2023 13:37 UTC
Last Modified: 11 May 2023 13:37 UTC
Resource URI: (The current URI for this page, for reference purposes)

University of Kent Author Information

Sawicki, Piotr.

Creator's ORCID:
CReDIT Contributor Roles:

Grzes, Marek.

Creator's ORCID:
CReDIT Contributor Roles:

Peeperkorn, Max.

Creator's ORCID:
CReDIT Contributor Roles:
  • Depositors only (login required):

Total unique views for this document in KAR since July 2020. For more details click on the image.