Typically windows and shiny metal surfaces reflect IR light, so you propably see your own "thermal reflection" - not the temperature of object.

Painted surfaces are ofc no issue.

The current gradient algorithm needs some improvement, but yes, with some effort you can use it to find cold-spots - and it needs to be cold outdoors to do this...