Most of my not-too-niche faves have already been mentioned. So, in the vein of newky's OP "in fact I often find the best new podcasts are ones that introduce me to an area I'm less familar with":
- Cunning Minx's Polyamory Weekly podcast about polyamory and the people who practice it, news, and issues that come up and general relationship stuff - because they're doing what the monogamous of us are doing but more of it, so actually more than you'd think applies to 1-on-1 relationships too - the things that make those work also makes other relationships work too. Its USA focussed but does have some international audience. I'm not poly (tried it briefly years ago, decided not for me) but I still find lots of interesting stuff there. Huge archive of episodes on all sorts of topics.
My Tech/Science-related additions:
- SE Radio - fantastic one for s/w developers (or if technically minded in this industry) if you want to keep up with different languages/techs/methodologies/architectures - totally serious mood, covers everything it means to be a rounded s/w developer or architect or in the software business, mostly in deep-dive interviews with someone from an industry, either a developer or architect/other technical role. I pick and choose my downloads based on interest.
Awesome Astronomy - I have strong interest in astronomy and know the guys who produce this one (UK based); the bloopers (included after each show usually) are sometimes the funniest bits, sometimes contains obscenities.
- BBC's Science in Action, the "Science" Magazine one, NPR's Science Friday
With all those you end up with a lot of the same stories analysed so I often skip episodes/segments of at least one of them where I've had enough.
- BBC Click (the radio show converted plus an intro/outro - not very deep, pop tech/little bit of science, but often-enough leftfield interesting stuff)
- Ubuntu Podcast, already mentioned of course; "Radio 4"-ish as alleged, but enough banter to keep it entertaining for me; also looking forward to Oggcamp again this year hurrah!
- BBC's Friday Night Comedy (though better when its the News Quiz not the Now Show which alternates with it between series)
You can also get TED Talks as an audio podcast - for the talks where audio-only works, and maybe there's a video version too.
I use PodcastAddict on Android, with the adverts, but nicer to use than the open source players I tried. gPodder on PC (but almost never listen via that). gPodder on my Nokia N900 did a pretty good job but gpodder-on-Android exploration attempts/incarnation doesn't seem likely to ever exist/any day soon - I think I read of some basic efforts towards it years ago). PA lets you use its own player with speed control so 1.1, 1.2, or 1.3x real speed gets you through the slower ones without the Helium-voice affect and especially for less technical content or slower speakers, to fit more in to my listening time.