Yes, the configuration seems to be saved in the memory of the SIM card since I was then able to connect to the network on Nokia N900. Or the configuration could be done on the network level (saved by the mobile network operator) but I think it is less likely.
Maybe with a SIM card reader and a good software, it is possible to configure the settings from a computer (but this would not be convenient when you are not at home or not with a laptop on you). If "dumb" / feature phone don't have this functionality, it could be a problem indeed like for Nokia N900.