It is designed that way. It is a peculiar gift, none other like it. It is the only sign gift that is the same type of sign that Jesus, Himself, was. Did you know that Jesus was a sign? Luke 2:34 calls Him "a sign which will be spoken against." Jesus was for "the fall and rising of many in Israel." IOW it is a double edged sign, having both a negative and positive side to it depending on who is responding. The disciples rose, while most of the Pharisees fell and wanted Him dead. And these signs are the only ones that reveal the secrets of the heart by the response they generate.
Jesus - Luke 2:35 "that the thoughts of many hearts may be revealed.”
Tongues -1 Cor. 14:25 "thus the secrets of his heart are revealed."
In 1 Corinthians 14:22 we see that "tongues are for a sign to the unbeliever." Without meaning to, there are so many Christians that see that and immediately think of the Day of Pentecost when the devout Jews turned to Christ after each heard the disciples speak in tongues, and erroneously believe it only means that tongues is a positive sign, even going so far as it was for preaching the gospel. But it is double edged. Remember there were others who mocked and said they were drunk. Verse 22 is showing the negative side to tongues, and verse 23 confirms it that if ALL were speaking in tongues and an unbeliever came in and saw them, they would say they were crazy. Today, the mockers use the words "gibberish" or "babble." So it confirms them in their unbelief, and reveals their hearts.
That is why Paul made restrictions inside the church, that only 2 or 3 speak in "gibberish" and immediately is interpreted. And if the interpreter is not there, then no one is to stand up and speak forth a message from God in just tongues. Because no man understands them. 1 Cor. 14:2
So it really doesn't matter that you can't test tongues. They are bonafide gifts of the Holy Spirit, so you can either accept their existence with the faith of a child, or speak against them. The test is actually about us, not them.