The appropriate directory's are the XDG Base Directory, check this page on arch wiki to move them correctly. You can also use xdg-ninja and it will scan for these extra directories on your $HOME and tell you what you need to do to move them to their appropriate location or not if these directories are hardcoded or cannot be moved.