You can click on the object or use a selection rectangle with shift key. You may also choose the object you want to deselect listed in the Toggle selection menu where you right click while pressing shift key.
See also Sweet Home 3D shortcuts.