Issue:
Using the "select all" characters hotkey moves the camera to a point of no interest to the user.
Cause:
There is no reason to assume the player wants to have his camera moved to the party leader when selecting all characters, as there are many more situations to "select all" than just gathering all characters at the party leaders position.
Suggestion:
Don't move the camera when selecting all, but only move the camera when the user pressed the hotkey twice in a short time frame (double click).
I would also argue, one of the most used cases is to "select all" when scouting ahead with one character in stealth, or when looting things.
Camera thus is already on the focus of the attention, and "select all" is being used to select the rest of the party, to quickly call them to the current location of attention. Moving the camera away from that focus is not useful.