This is the reason serialized player saves are trash. There's no good way to modify/remove information without causing issues (as far as I'm aware). Just add a condition in the "start()" method (or whatever you call when a player logs in) to migrate your Skills.java:
Code:
Player:java
public void start(...) {
...
skills.migrate();
...
}
Code:
Skills.java
public void migrate() {
if (level.length == 26 && xp.length == 26) return;
newLevel = new short[26];
newXp = new double[26];
System.arraycopy(xp, 0, newXp, 0, 25);
System.arraycopy(level, 0, newLevel , 0, 25);
xp = newXp;
level = newLevel;
}