I just had my yearly exam at the OB/GYN and brought up essentially the same issue. Was told yes, kegels are the answer but it takes practice to isolate the correct muscles in the pelvic floor. She recommended physical therapy with a pelvic floor specialist (who knew such a field existed!) for one or two sessions to be properly trained on the exercises and then to commit to a lot of work to strengthen those muscles, cautioning that it will not get better without putting the work in.
However, I’ve had three kids and so the underlying cause for your situation is quite possibly different than mine.