You need to read the whole chapter. Verse 1 tells us the Lord is the one who wrote on the stone tablets:
"
1Then the LORD said to Moses, “Chisel out two stone tablets like the originals, and I will write on them the words that were on the first tablets, which you broke."
Exodus 34:1
So we know 'He wrote' in Exodus 34:28 is referring to 'the Lord' in verse 28, not 'Moses':
"
27The LORD also said to Moses, “Write down these words, for in accordance with these words I have made a covenant with you and with Israel.”
28So Moses was there with the LORD forty days and forty nights without eating bread or drinking water. He wrote on the tablets the words of the covenant—the Ten Commandments."
Exodus 34:27-28
So your theory that Moses just got it wrong in Deuteronomy is wrong.