Former, but not just former. Using a literary term for it gives it a flavor of . . . something. Maybe a touch of irony, as is often the case when inflated or fussy or formal language occurs in ordinary speech. The word attracts a little emphasis, and that may make it seem slightly ironic.
Still without looking it up, I would decode it as “erst” = first, oldest, and “while” = time; interpreting it as belonging to a former time, and not literally first or oldest. (Note, this is not true and verified etymology, just my best guess.) There is that sense of relating to a past period or era, and not just something superseded.
I would tend to reserve it for relationships of some sort; for example, his erstwhile lover, your erstwhile employer. I would not refer to her erstwhile car or my erstwhile residence. Well, maybe the latter.
Ok, let’s see what Webster says.