This is only in some instances, I think maybe when I change a setting although not every time. I keep the window at its smallest size so when I change a setting and then go back I have to manually drag the window smaller again.

Thanks Philip!