The amount of rubbish that can acumulate in the three aforementioned directories is amazing. I don’t have any compulsive-obsessive disorder, so I can live with these three directories being full of a lot of things and don’t care… except when things break because it happens that some arcane configuration file is outdated… (GNOME font configuration cough). Really, I sometimes would love having an Android-like “Delete user data” button on GNU/Linux…
Is there any sane solution out there that might deal with these things? I sort of root for AppImage/XDG-App-like solutions in the hope that these might help sorting out the dotfiles/dotfolders hell. How do you guys manage this? Do you “purge” everything once and then? But I feel that’s not the idea of having a separate /home partition.
neuro@intrepid:~$ du -sch .config .local .cache 4.2M .config 496K .local 12M .cache 16M total
Every time I do a system version upgrade I move the contents of those directories to a backup location. It usually only takes me a half hour or so to reconfigure the user interface of the newest release and a few programs to the way I want.
I keep the backups of configs for a while, but rarely need them.
The times I’ve taken that route I end up requiring a lot of time to reconstruct things as I like them… 
I should try again one of these days.