I can’t say that you’re alone, and I’ve certainly made some wrong estimates during my years as a freelancer. One of my dependable rules of thumb was that every job has its problem.
But because of the way I tracked my time on task, I got good enough at estimating my jobs that I could come in within half an hour on a job that was going to take, say, 80 hours. I always built in some leeway (proportional to the size of the job) for what I called “admin,” which took in those odd hours of overhead.
And there were a few times that I simply wrote off the extra time for the sake of client goodwill.
An important point was that I wouldn’t estimate work on any job sight unseen. (Learned that the hard way.) I had to get a good sense of it before I’d quote my rate and my time.