Mary was not a single woman after giving birth to Jesus. She was a married woman. The Jews use Mary's marital status as a married woman to accuse her of being an adulteress, because her marriage was not dissolved after childbirth.
Matt. 1:19 However, because her husband Joseph was righteous and did not want to make her a public spectacle, he intended to divorce her secretly. 20 But after he had thought these things over, look! Jehovah’s angel appeared to him in a dream, saying: “Joseph, son of David, do not be afraid to take your wife Mary home, for what has been conceived in her is by holy spirit. 21 She will give birth to a son, and you are to name him Jesus, for he will save his people from their sins.”
Joseph did not abandon his wife.