Kerosene heating works well. Many "oil" stoves are used in cottages for heating and do a nice job. It takes a while for the main housing to get up to temperature, then turn it down to a minimum flame and it's good for hours. Use an old furnance tank outdoors and feed the stove by gravity. Make sure the fuel tank is sound and that you can meet any venting requirements (chimney).
Best bang for the buck is a overhead forced air heater and a 100 pound propane tank outside. If you have gas near by you could run a line to the shop. I went through the exact same ting you are going through. Wood stove then several types of portable heaters then my overhead gas heater.
+1 I worked with a guy years ago who had a neighbour call the Ministry of the Environment when they saw his fuel tank leaking. He had to have all the soil removed around his shop as it was contaminated. Not sure what the final bill was.
We converted to gas, and still have the old oil furnace sitting in the barn. It's yours, if you want it.
