Peter and the other apostles, along with the prophets, formed the rock foundation. And I read those verses as Jesus delegating His keys and authority to the CHURCH He was/is building on that rock foundation....as well as Himself as Rock (chief cornerstone). Binding and loosing according to scripture has to do with forgiving (loosing from) debts, or not forgiving for a time if the person needs to be given over (bound over) to the Lord's discipline for a time. It also has to do with healing and deliverance, which gift/authority/power is also given to various parts of the Body/church according to their spiritual gifts (or even if God chooses to use any member that way at a given time even if they aren't particularly gifted that way normally).
I arguably think Peter was unspoken leader of the apostles though, he was being singled out by Jesus here, but not declared to be so in a formal way, just tacitly, in simplicity, and we do see him standing up to take the lead at times...as led by the Spirit. God's ways are not man's ways and I don't like seeing those verses being used to justify man's ways of making an institutionalized religion out of the Way of Christ.