No, stars such as our sun work on thermonuclear fusion of hydrogen to form helium. Our sun isn’t big enough to fuse to oxygen yet, but it still produces enormous quantities of heat. Combustion is not the only method of producing heat.
If you are talking about the presence of heat rather than production, heat is the oscillation of atoms (mainly contributed to by electrons) in a particular energy band.
Let’s not forget the transfer of energy in the form of radiation. This only requires radiation (some form of electromagnetic wave) and a source that receives it.
@Shuttle128 Heat can be transferred by infra-red electromagnetic waves, but that is not heat until it interacts with the receptive atoms (as you mentioned). However I think this question is aimed at heat production rather than transmission since @Blobman refers to conventional combustion.