As its name suggests, the PSP -- or PlayStation Portable -- is a smaller, lighter version of Sony's PlayStation. It's not as powerful as the PS2 or PS3, but is more powerful than the original PlayStation (PS1), and it has a built-in screen and speakers so you don't have to plug it in to anything to use it.
As with the full-size PlayStation, which can play DVDs and audio CDs as well as PlayStation game discs, the PSP is more than just a gaming handheld. The PSP plays games in UMD (or Universal Media Disc) format, but UMD format is also used for movies and other audio-visual content, and the PSP can play those, too. And if that isn't enough, the PSP can also play audio, video, and still image content off of Memory Sticks (Memory Stick Duo and Memory Stick Pro Duo are the compatible formats).
Unlike the PlayStation or PlayStation2, the PSP comes equipped with Wi-Fi, so you can connect with other players wirelessly and, if your firmware is version 2.00 or higher, get online to browse the internet. Sony has also announced an add-on camera and a GPS device for use with the PSP in some regions, expanding its capabilities even further.
Basically, what all these fancy specs come down to is a kick-ass handheld gaming machine with amazing graphics for its size, that can also play movies and music, display photos and image files, surf the web and, with every new firmware release, even more cool stuff. Phew!
As for whether or not you should buy one . . . well, you'll have to decide for yourself, but I've never regretted buying mine.