Shops I've bought from in person:
Borders (a few manga volumes).
Anime conventions (tons of manga)
Shops I've bought from online:
J-list (mostly magazines/ artbooks, a figure, various small things)
Play-asia (some games, a figurine, a few various things)
Himeyashop (Umineko visual novel(s))
Justmanga (some manga, very large selection though average prices)
Akadot Retail (some magazines I couldn't find elsewhere)
RightStuf (currently in the process of shipping, haven't used before but it's definitely cheap, especially with "membership" + when they have sales. Also, international shipping is pretty cheap compared to some other places. Good for manga and anime.)
Very likely to use in future: (haven't at present)
Cdjapan (various - cds, dvds, books)
Various shops in Japan
Mangagamer (probably just higurashi visual novels)
.........................
But yeah, with the collectors mindset: Owning things is much more satisfying than having hard drives full of shit. Still, having both is a bonus.
Also, I much prefer printed media (eg manga, artbooks, magazines etc) in book form than in digital... which is another big incentive for buying (the stuff I like enough anyway). ^^