I am now able to run some initial Rockbox code on this player. I'm using the DFU mode for this, this allows you to send some code to run from internal RAM. Since the code only gets copied to RAM, it's a quite safe method of experimenting with nearly zero risk of bricking the player. I am able to control the display (showing a little rockbox logo and some debug texts), backlight, radio, button LEDs, battery charger and read the hold/power switch, real-time clock, USB and battery voltage. This is nowhere near a state where I can play mp3's but it's a good start.
A big stumbling block to further development will be access to the filesystem on the NAND flash, because the original firmware uses a proprietary software layer to read/write from it. Possibly I can reuse code from the openiboot project for this. Also I can't seem to get any response yet from the touch key controller, the Melfas MCS-3080, there seems to be basically no documentation on this chip.