Should the Sweet Home (installed version) be deleting its temp files after a user exits out of the program?

Yes, it should (and it does under Mac OS X and Linux), but I discovered recently a bug in Java that prevents the deletion of some temporary files under Windows.
I reported it to the Java bug database in June but I'm astonished they didn't make the bugs public yet:
Bug #6962459 - Can' t delete zip files accessed with jar protocol
Bug #6962458 - Files tagged with deleteOnExit are not deleted when accessed with jar protocol

I could program a workaround but this would imply big changes to get something clean, therefore I prefer to wait and see what Oracle is going to do with these bugs. Once they are made public, I will ask users to vote for them.

Sorry for the inconvenience.