I entered the Palace Residence (as suggested by the quest) and left, no problem. In the same stay in Caer Lem, I entered a second time in the residence and the exit box vanished. I tried short rests, long rests, and reloading an earlier save (earlier save was also in the Palace Residence) - none of that restored the exit box.
go to exit and try to rotate map - it helps me
yes, had to move the map around and click about the door, but finally got the exit square to show up. frustrating, but eventually you can get our of there.
Madcat451
This is probably the area of chapters in development. You must exit to keep enjoying to game.
I had the same problem. Just move your group right before the door (where the exit square should be) and then back again. When you exit the squares place, it will appear.