Thread 4811 and the Sunlight simulation blog post may help shed light on your issue.
My understanding is that in Aerial view the sky texture is not rendered. Instead the sky colour is simulated based on location, orientation and time.
Shadows disappearing when rendering in virtual view sounds strange. Colours rendered are influenced by the sun position and therefore will get a warmer glow in the afternoon. Did you adjust compass and photo rendering time correctly? Something with light sources?
ok
PS: Virtual rendering showing warm sunlight, sky and shadows:
