Last time this happened, I had not made the minimal Perl version it required explicit. Specifically, the // operator was introduced in Perl 5.10, so the code naturally would not even compile on versions 5.8 or 5.6, which cpantesters.org still includes.
This time? One test wrote a data structure as a Storable file. Since I don't want the tests to need to be able to write to their own test directory (which might in fact be fine, but why require this if I don't have to?), I created a function which would write these files if they didn't exist, but include existing versions in the distribution. This way, I could delete them manually if I needed new content, allowing the altered test to create its own files, but would still be distributing already-written versions.
However, I used Storable::store instead of Storable::nstore. The purpose of Storable::nstore is to use a special format that can be transfered between different environments, like 64-bit Linux to 32-bit Windows. Which is, of course, what the large array of testing machines in cpantesters.org contains.
Oops. Here's hoping nstore fixes it :-P