My first karate teacher started at age 5 to study his art. He was very talented and was a 6th dan black belt when I knew him. However, my second teacher started studying at the age of 19. He achieved his black belt in about 3 years, but never made it past 3rd dan due to personality issues. He also was talented, but he was never as smooth as the first man.
I know a lot of folks who started studying martial arts when they went off to college. If they worked out several times a week and worked hard at learning the katas and the various blocks, kicks and strikes, and if they worked on their humility, it was very possible to attain the black belt by the time they graduated, so in about three to four years.
In fact, I started studying martial arts in college. I went to three classes a week, and sometimes a fourth special class called 1000 technique night. I ran five miles daily, and I lifted weights to gain strength. I tested for my black belt after three years of study, but was injured during the test and so was unable to continue.
So, yes, it is very possible to attain the rank of black belt even if you start in your 20’s. You might not get to the higher levels of black belt, but you will still have amassed the skill to be accepted as a practitioner.