Chosen Solution

I have tried all recommended shutdown methods but keeps coming back on!

What the… Just remove the back cover, then disconnect the battery. Make sure to use plastic or nylon tools iPhone 4S Battery Replacement

If the phone won’t stay shutdown, look for debris or damage in the dock connector as this may be the source of your problem and not the battery.