Yeah, I didn’t learn to swim till my senior trip. I was drunk and jumped into the deep end of the pool with all my clothes on (except my shoes and coat). My friends dared me and thought I wouldn’t do it but I figured I could float and so I jumped in and floated and then they told me to paddle and I did and I found I could at least swim backwards. Then I did it while floating on my stomach. I was like you. Too scared to let go of the edge. I’m not suggesting to get drunk. That was a stupid idea but I realize, I was missing the fear at the time and so I was able to finally do what I couldn’t when I was sober. Since your partner is such an awesome swimmer, trust that he can pull you out if you need rescuing. My husband was a swim instuctor at Parris Island and he taught me to tread water and to dive from a high height. This may sound silly, but try one day holding onto the edge and inhale deeply through your nose and exhale slowly as well. Do it about 4 times and then hold the last deep breath and just put your face in the water and time how long you can stay under. Provided you have healthy lungs you will find you can hold your breath a long time and longer each time you do this. Of course there is a limit. Your brain needs air. Anyhow, after a few times of doing this and sink your body down and relax in the water like its a bathtub.
Then start to let go of the edge and tell you SO not to grab you unless you start to panic. Once you know you can hold your breath for about a minute than face down in the water and swim alongside the pool with your partner on the outside of you. If you can hold your breath for one minute relaxed you should be able to hold it half a minute swimming.
Another thing is learn to tread water poolside. Once you know you can actually support your body by just kicking your feet in an upright position than you know if you start to sink by trying to swim that treading water can lift you up to where you can get air and it will help ease some of the panic.
Even if you can’t float it is possible to swim so long as you keep the motion of paddling.