The PSP and PS3's XMB, or Xross Media Bar, was developed to be relatively easy to use with buttons. I say "relatively," because some new users found it a little confusing, especially on the PSP's smaller screen. It was based on a horizontal row of icons (or "bar") that each revealed a sort of drop-down menu of options, which might change depending on what you happened to have contained on your memory stick and whether or not there was a UMD in the drive.
Though the PS Vita has the same buttons (as well as a slew of other controls) Sony chose to develop a whole new user interface for the handheld, based on its touchscreen. While it's not impossible to imagine using an XMB-style interface with a touch-screen device, and it would actually be interesting to compare it to other touch-interfaces out there, basing the interface on the way people interact with things they can touch, rather than just making a touchable version of the existing UI, is likely to result in a better user experience.
When starting up the PS Vita, the user is presented with an array of round, bubble-like icons, perfectly sized for touching. The user can rearrange these icons however they like, and if they end up with too many to fit on the screen, they can simply swipe vertically with a finger to move onto the next screen full of icons, and the next. There's no word yet on whether or not users will be able to combine icons into folders in order to reduce the number of screens they have to swipe through, like iDevice users can, but it's possible such categorization won't be necessary. The number of items installed on the PS Vita's onboard memory will likely be relatively small--more games and save files would be stored on memory cards, so swapping to a different card would create a different selection of icons on the PS Vita's main screen.
The PS Vita will come pre-loaded with a variety of programs, and there will be others available for download-. Each program is represented by an icon, and tapping the icon will start up the program. Tapping the icon for a game will bring up the game's "LiveArea" (actually, there seems to be some confusion about whether the term "LiveArea" refers to the whole UI, or if it specifically means the screen that pops up when you tap a game icon). This is a screen containing information about the game, any trophies achieved, and available updates, plus links to game-specific resources like web sites, message boards, and so on.
No Buttons Allowed
One aspect of the PS Vita's UI that gamers will either dislike or not care about at all is that it can't be navigated using buttons or the d-pad. It's touch-navigation only. Because the UI has been designed specifically for touch navigation, though, even if you could use the buttons, it would likely be awkward. Not impossible, and maybe even not too bad, in the same way that navigating the PSP or PS3's XMB would be possible and not too awful to navigate with touch.
But in much the same way that iDevice users quickly got used to only having one big button on their devices so they had to do almost everything by tapping icons, until most of them would not want to go back to multiple buttons, I suspect that once PS Vita users have played with the UI for a while, they'll be much happier with it than anyone ever was with the XMB.