Deep Learning the Colour Patterns of Coral Reef Fishes

Animal colour patterns serve a variety of ecological functions related to e.g. signalling, camouflage, mimicry or defence against predation. Studies have long used colour patterns as a model system for understanding evolution since they provide exceptional access to phenotypic diversity and coral reef fishes display a rich diversity of colour patterns and are phylogenetically diverse, making them a unique group of targets for studying colour patterns. Nevertheless, the phenotypic diversity in reef fishes is not explainable by their genetic diversity and the evolution of colour patterns and their significance for diversification are still poorly understood. Colour pattern is a phenotypic trait that involves colour, geometry, morphology and vision systems. With the well-established techniques for encoding genotypes, we are now limited by our ability to encode this complex phenotypic trait objectively and quantitatively. In the commonly used colour pattern quantification methods, colour and geometry are not simultaneously addressed and colour pattern diversity is confounded with body shape diversity. We present here a workflow that disentangles colour pattern from body shape using morphometric methods and automatically extracts colour patterns across a broad taxonomic range. The workflow consists of two parts, (1) machine-assisted landmarking using community-sourced fish images and (2) colour pattern quantification using a deep perceptual metric. We show that using deep-learning-based techniques, colour and pattern can be concurrently assessed, allowing new possibilities in research that will enhance our understanding of the complex evolution of colour patterns.