Many people seem to confuse "off" with "standby". Just because the screen and LEDs are not glowing does not mean the phone is "off". STANDBY is the state where the phone is still making periodic GPS fix and cell tower communication. A phone can stay on standby for several days, "dumb" phones (but with GPS) could be on standby for a week plus (my Samsung back in 2006 could easily go 10 days on standby).
When a modern phone is powered OFF it is still somewhat active--this is because it there is no hard disconnect for the power. When you are turning on the power you're pushing a button on the phone--there is some chip in there waiting for that contact to happen. However, in the OFF state phones are using very, very, very little power... there is no radio or GPS activity, etc... they can last in this state for weeks, possibly months.
Many smart phones today you cannot remove the battery... but once it's dead it is dead. These are not magical devices and there is no secret backup battery.
Anyone saying that a phone without a battery, or a phone with a very dead battery, can be tracked is doing nothing but spreading FUD.
The FBI, the NSA, whatever, cannot track a cell phone with no battery or a dead battery--it's just an inert pile of plastics and metals at that point. They can track a phone that is standby.
However, I would not put it outside the realm of possibility that even when the device is in the "OFF" state it does not wake up every few days to check GPS or cell position--It is highly, highly unlikely and this would be something added either as a hack or a backdoor or something... not what a manufacturer would build in on purpose.
BTW, the coin cell batteries found in old PCs and electronics were usually to run the RTC--Real Time Clock--and would provide small current for configuration (BIOS) memory that was volatile. They are rated at 3V (cell phone battery will be 3.7V) and cannot supply anywhere near the kind of power needed to run the GPS or cellular radios in the phone for any length of time, I'm talking seconds or a minute at the very best. Virtually all phones you would have today do not have one of these batteries in them anyway because the phones do not use an RTC or volatile memory.
I've been an electrical engineer working for semiconductor manufacturers making chips for these devices for over 10 years.