Thought I'd start this off and see what others thoughts are on the subject.
Anyone who has or has owned a PSP (any) or has knowledge on this kinda thing (or who would just like to comment on this) is invited to share their thoughts, memories and rants etc. on how the platform as a whole hasn't really come all that far from the awesome original.
For my own view, I'm heavily disappointed in Sony on this one. They could have done so much (still can if they get themselves together) but chose to (on the whole) almost completely ignore the needs of the market and what their customers wanted out of the device. Take for example, the fact that the PSP is now in its (what is essentially) 4th outing in the Go. From the original 1000 series the only notable new/improved features that come to mind are: bluetooth, slider format, a large amount of inbuilt storage off the bat and a slightly better screen. The TV out connector can't be defined as a useful addition as so few even use it for much if anything at all - in reality all it does is allow you to output a heavily zoomed video of a game onto your TV.
Moving onto the direction I think Sony should take the platform in, I'll use the concept render below as a base for what I propose they ought to do.
Considering how many loath UMD's but also taking into consideration how many now own their games for PSP on said disk Sony could offer a UMDless version (with a UMD drive as an accessory using the USB 3 connection). On the subject of connectivity, 'N' class wifi (backwards compatible naturally) and bluetooth (vers. 3 I think now) need to be included in addition to USB 3 therefore not only providing the PSP with faster transfer speeds to and fro but the ability to charge quite considerably from the USB cable alone so that forgetting to bring your AC adaptor with you isn't an issue any longer. Other things like the dual joysticks I'm not gonna talk about here cuz it's visible enough in the render already. Oh and to make sure in the power stakes the PSP keeps up, since Sony have a contract with nVidia already for the gpu chipset used in the PS3 they can utilise that and have a tweaked version of the Tegra 2 System-On-Chip (SOC) chip for the PSP2 as I'm calling it atm.
As for the screen, it is pretty good already (on the original) but with new products like Apple's iPad, the iPhone and the new 3DS (which incidentally is using a version of the Tegra if memory serves right for its 3D effects) rising as popular games systems especially for casual use as they're touchscreen devices, the PSP kinda needs to up its game. All things thought, Sony would be smart if they sourced IPS panels for the PSP and made the whole panel a capacitive touchscreen.
Thinking about how Sony have implemented the XMB on the PSP, multitouch support for the browser etc. and touch support across the XMB would prove a massive hit in scrolling through your list of music for example. The addition of a touchscreen would also prove useful in attracting more casual gamers in addition to those hardcore gamers already on the PSP as Sony could release more puzzle type games for example (just a thought) to utilise the touchscreen. A must would be for the screen size to be at least 5.2 inches with an aspect ratio of 16:9. Combined with the slider format, the unit could be shut and the touchscreen used exclusively to navigate the XMB.
Keeping most everything else the same as far as I can think e.g. the lock switch, wifi switch (which could be made in a double slider - nudge one way and it activates/deactivates wifi, nudge opposite way to activate/deactivate bluetooth with the switch moving back to the middle after you release it naturally). One last useful feature, especially for media heavy users (like me

) would be the inclusion of a SDHC compatible memory card slot in addition to the inbuilt storage of course.
For reference this is the render I would be extremely hopeful for Sony to make with a few modifications:
Hope I've got the ball rolling for this, I'd be genuinely interested to hear if there is anyone else like me who is saddened by the lack of interest Sony has been showing the PSP platform for the past few years.