The cheapest approach is not a particular fuel. It is heating only the people who are there, only when they are there. Radiant infrared systems do this naturally: they warm bodies and surfaces directly instead of trying to fill a draughty medieval volume with hot air. Switch on five minutes before the congregation arrives, switch off the moment they leave. No hours of pre-heating.
Gas convection can look cheap on paper per kilowatt-hour, but it runs for longer, heats unused space above head height, and loses heat through every gap in the building. Once you factor in actual run-time rather than unit price, radiant electric heating often costs less to operate in a typical church.
Zoning helps too. If your midweek group uses only the chancel, heat only the chancel. A decent infrared system lets you do that without walls or partitions. Combine targeted heating, short run-times and a simple timer or thermostat and you will keep costs as low as they can realistically go.