This is a toughie. I really like all the Handheld Game Systems I've played. I can't just choose one. =/ But I guess the Gameboy Color...only because I had some of the best memories with it.
My favorite is the DS Lite, because of the touch screen gameplay.
Other Nintendo handhelds like GBA and GBC are my other favorites, basically because they're the classic ones.
Both PSP and DS Lite are great consoles, both have some excellent games I would get the console for.
However, due to my Tekken fanboyism, I lean slightly more for PSP. Come on, I get to have my favourite game Tekken 6 on the go? Can't go wrong with that.