I have several heaters. My house uses fuel oil, but I mostly just use space heaters and I'm fine. Of course, I don't mind it cold and most of the time I have the temperature set to about 59 (I do turn it up if I have company!). I'll set the furnace to come on at 50, and so it usually doesn't even come on.
In the living room I have my infrared heater. I bought it last year and paid $130ish for it. It works wonders. I'm not sure if it's because it's a special type of heat (I kinda doubt it) or more because it aims the hot air at the floor instead of the air. But I love it and it's safe to use. It will usually keep the living room/dining room around 58-62 degrees.
In my dining room/library I have a DeLongi oil-filled radiator. I use this 24/7 in the coldest part of the winter when it's 0 or below outside. Otherwise I just use it in the evening or early morning when I'm going to be at the table for a bit and it's kinda cold. It just fits under the table so I'll slide it under there and it heats up the whole table, and I move it when I'm not going to be using the table.
Upstairs I have an old Honeywell hot air heater that my mom got me years ago. I use this one in my bedroom at night.
A lady at work got 2 of the infrared heaters to heat her trailer. She was upset over the electric bill, but I think that was more because she was using them to heat the trailer to the 70s.
I wouldn't use the infrared heater as your main source. But as a supplement they work very nice and I think I have saved a ton of money using them. This is my 4th winter here. I spent a ton of $ the first year on fuel oil - probably nearly $3,000. The next year I got a programmable thermostat, and I spent probably $2,000. Last year I got the infrared, and combined that it was a warmer winter, I spent less than $1,200 including the cost of the heater. This year I have spent about $300 on fuel oil, and my electric bill has increased about $40-50 a month while I'm using the heaters - not really much when you consider the money I'm not spending on fuel oil.