The gathering is to 'heaven'; the difference is that the heavenly dimension is going to be revealed right here on earth. By your thinking, it shows you still do not understand the two dimensions of existence written of in God's Word. God Himself in final is going to live right here on earth, and He is a Spirit. So how is that possible?
Mark 13:26-27
26 And then shall they see the Son of man coming in the clouds with great power and glory.
27 And then shall He send His angels, and shall gather together His elect from the four winds, from the uttermost part of the earth to the uttermost part of heaven.
KJV
The Matthew 24:31 version of the above is about the 'asleep' saints Paul said that Jesus will bring with Him when He comes. But the above is about Christ's faithful saints still alive on earth when He comes. The "last trump" change "at the twinkling of an eye", will happen first, the heavenly dimension opened up on earth for all to see, and all changed at an instant to the spiritual type body. It will then be in that "spiritual body" how Jesus will gather His faithful on earth to the asleep saints, and they all then go to Jerusalem with Him. Simple, if one pays attention to the Scripture as written, and believes what it says.