However, I believe I'm going to need some sort of custom firmware to get it working with emulators and that.
After chewy pointed me in the direction of ChickHEN, I had a look at that (although I really had to dig around to get it to work, and finally found a method on the PSPHacks forum that worked for me). However, what I didn't know until after a bit more digging around is that ChickHEN is a temporary patch that sits in memory, which is therefore lost when the unit is powered down. Ideally I'd like a permanent solution so I didn't have to keep running the ChickHEN trick every time I powered the unit on.
After googling around a bit, I found out that some PSPs are easier to mod than others, and certain motherboards aren't moddable at all. I've checked out my motherboard model number and it appears to be a TA-088 v3 (which seems to be one of the supposedly un-crackable boards).
Am I stuck with ChickHEN, or is there a way around it? I'm not bothered about playing PS1 or "backups" of PSP games; I just wanna play with emulation

Footnote: I say that ChickHEN works, but I'm not having any luck loading some Emulators. Ones I've tried are:
SMSPlusPSP 1.3.1 - Application loads, but when I select any ROM file, the PSP freezes and shuts down.
SNES9X Euphoria R4 - Freeze and shutdown
NesterJ 1.12Plus 0.61 RM - Seems to work
PSPHUGO-v1.3.1-fw5X - Seems to work
Unofficial PicoDrive v1.35b - Seems to work