I try to be stronger, I can barely control it at times if I even can
Hey, it's OK. God knows you and what you have experienced. Just ask Him to help you. He takes everything into account. Look at this:
Psalm 87:5 And of Zion it shall be said, This and that man was born in her: and the highest himself shall establish her.
6
The Lord shall count, when he writeth up the people, that this man was born there. Selah.
In other words, when God is looking at His people, He will count as well the past of His people, where they were born and under what circumstances they lived, or mental problems, etc.
Zion--His children have given their hearts to Him, or they would not be of Zion. He takes us where we are. We knows our weaknesses, and will give us strength to overcome. This may not come immediately, sanctification is a work of a life time. That you are being bothered with oaths is important. God is bringing it to your attention, and wanting both of you to work on it together. Try a little every day to remember not to do it, and pray when an opportunity comes to do so.
I had a similar problem, but with smoking. Everyone told me--God will take it away, just ask. Well, It didn't work, and that was because of me weak in the flesh. But we can up with a plan. I went from smoking to vaping full concentrated e juice. Slowly over time, I reduce the concentration, then finally got down to 0% nicotine, and after couple days, I was done. Haven't vaped in over 5 years or smoked in over 10.