The main purpose of geographic coordinates is to determine the sun elevation above the horizon to draw shadows and reflections as it would happen in the real world.
Moreover the GPS is not accurate enough to take measurements for a single building, you shouldn't worry about such fine alignment.