Small progress.

Using this settings:
-Dsun.java2d.opengl=true
-Dsun.java2d.d2d=false

leads to this.

Using just settings:
-Dsun.java2d.d2d=false

shows error message that there is problem in 3D rendering system initialisation and SH3D is closed.

Using settings:
-Dsun.java2d.d2d=false
-Dcom.eteks.sweethome3d.no3D=true

finally makes the SH3D working and no UI glitches so far. Disadvantage is the missing 3D overview and rendering possibility.

So it definitely seems like problem in Swing components rendering using Direct3D.