I think they’d be alien enough that the types of thinking they would have would be alien to ours, so that they would have different taxonomies of concepts. So from their point of view, religion and spirituality would probably not map from their thinking into our thinking directly. After all, on earth, not only are their non-humans who most/all have no freakin’ concept what religion or spirituality are in modern terms, but even a few hundred years in the past, humans would not mean the same things when using those words, and would not think about them the same way. Even if you travel between communities in the same town in the USA, you will find significant differences in how people relate to those concepts.
So, people who try to understand the aliens may or may not think of what they try to understand about the aliens in the context of those humans’ own ideas of religion and spirituality, and find ways to use those words in regard to the aliens, but I am sure it won’t be a completely accurate match or mismatch.
Also, there are human spiritualists of various flavors, not including delusional folks, nor charlatans or fools, who believe they have experienced spiritual contact with aliens. Just sayin’.