Pré-Publication, Document De Travail Année : 2025

PhyloLM : Inferring the Phylogeny of Large Language Models and Predicting their Performances in Benchmarks

Résumé

This paper introduces PhyloLM, a method adapting phylogenetic algorithms to Large Language Models (LLMs) to explore whether and how they relate to each other and to predict their performance characteristics. Our method calculates a phylogenetic distance metrics based on the similarity of LLMs' output. The resulting metric is then used to construct dendrograms, which satisfactorily capture known relationships across a set of 111 open-source and 45 closed models. Furthermore, our phylogenetic distance predicts performance in standard benchmarks, thus demonstrating its functional validity and paving the way for a time and cost-effective estimation of LLM capabilities. To sum up, by translating population genetic concepts to machine learning, we propose and validate a tool to evaluate LLM development, relationships and capabilities, even in the absence of transparent training information.
Fichier principal
Vignette du fichier
2404.04671v3.pdf (7.11 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04880495 , version 1 (10-01-2025)

Licence

Identifiants

Citer

Nicolas Yax, Pierre-Yves Oudeyer, Stefano Palminteri. PhyloLM : Inferring the Phylogeny of Large Language Models and Predicting their Performances in Benchmarks. 2025. ⟨hal-04880495⟩
0 Consultations
0 Téléchargements

Altmetric

Partager

More