It sound like it could be the beginning of osteoarthritis. Or, maybe not.
As you know I’m not a doctor, but regarding the baby aspirin, I really doubt the 81mg dose will be a therapeutic dose for inflammation or pain in an adult. That dose dose affect platelets to thin the blood a little, but if you think in terms of fever or pain, that dose probably wouldn’t touch those symptoms in an adult.
You certainly can try the low dose (assuming there is no reason for you to avoid aspirin) see if it helps. If it doesn’t you could take three pills see what happens, or try a regular dose 325mg pill. Typically the dose for pain and inflammation is two 325 pills, but start low and see what happens if you go the pill route. Remember to take with plenty of water and a little food. You might try taking Aspririn or NSAIDS at a normal adult dose for 3–5 days to see if you can just knock down the inflammation similar to how a short term prednisone Rx is used.
My MIL has fairly severe arthritis in her finger joints, and once every 5 years or so she takes two weeks of prednisone and it helps for a while. She takes NSAIDS regularly too.
Try magnesium spray on your fingers that hurt three times a week. It’s cheap and magnesium is absorbed well. Don’t over use on large parts of the body, there is a such thing as too much magnesium. This is similar to using epson salts.
I would get your vitamin D level checked if you don’t do that already. I know I know, typical JL answer. I was having hip and elbow joint pain for the first time in my life last year, and I tested my D and it was very borderline, so I started taking it religiously again to get my number up, plus eating healthier in general. The pain after several months went away. I can’t be sure what helped, but I do think the D might be a factor.
Also, check your calcium level and magnesium.
Your doctor might very well Chech D and calcium regularly, and you can just look at your most recent tests.