I don't think that's documented anywhere, but here's how you can do it (for the default skin):

.defaultSkin span.mce_sub {
    background:url(./path/to/subscript.gif);
    background-position:0px 0px; 
}
.defaultSkin span.mce_sup {
    background:url(./path/to/superscript.gif);
    background-position:0px 0px; 
}

Both those icons are of course 20 x 20 pixel images.

Put that code into the "tiny_mce/themes/advanced/skins/default" directory in a file named "ui_custom.css".

In your editor config, add this line:

    'skin_variant' : 'custom',

Now if you are using a different theme, simply replace ".defaultSkin" in the CSS file with the correct name. For example:

The Default skin = ".defaultSkin"
The Office 2007 skin (o2k7) = ".o2k7Skin"
etc....

Notice that the correct name is "skin name" + "Skin".

Hope this helps......