Yep, but to the Twelve specifically He said:
"If you forgive anyone’s sins, they have been forgiven them. If you retain anyone’s sins, they have been retained.”
To Peter specifically He said:
"I tell you that you are Peter [...] I will give you the keys to the Kingdom of Heaven", which is about giving him the authority to make decisions on earth that will be reflected in Heaven, because He continued saying to him, "and whatever you bind on earth will have been bound in Heaven; and whatever you release on earth will have been released in Heaven.”
"Tend My sheep"
and so on...
The authority given to the Twelve by Jesus was passed on to their successors. Matthias was the first to succeed by taking up Judas of Kerioth's office after his death. It's in the Book of Acts. This apostolic succession has continued to this day.