Therefore, only those who understood the words God used would be able to follow God's teachings. What happens to the rest of the people?
Just an idea to consider!
The short answer is that Yahweh will judge everyone on what was in their hearts.
It does not matter what they knew or understood. If they were truly trying to be decent life forms they will be accepted by Yahweh. He will judge. If you believe that Yahweh is “good” then you should have no problem accepting His judgment.
Relating to The Tower Of Babel:
That is exactly what happened! God chose to communicate His plan with one man. Abraham and his descendants, Israel and those who "spiritually" belong to the the group "descendants of Abraham".
This was relatively shortly after the Great Flood and after the time of the Tower of Babel.
Through Abraham and his descendants, Yahweh would keep in touch with the world until the time of the Gentiles, now, when everyone has access to Him.
After the Flood a man named Nimrod became powerful and built a great city in Mesopotamia. He declared himself a god and had an incestuous relationship with his mother. The people that still remembered Yahweh killed Nimrod and cut him up into pieces.
The life of Nimrod is the founding story behind most pagan religions in the Middle East and Europe.
The most famous name from antiquity after the flood, from the pagan perspective, was the Biblical character Nimrod who was responsible for the events that led to the division of mankind into the hands of the “sons of God.”
Nimrod is credited in Genesis as being “a mighty warrior on the earth” and “a mighty hunter before the Lord.” He is known as the first “empire builder” and through force of conquest he established his empire over the whole of Mesopotamia.
According to the Bible he first ruled from the cities of Babel, Erech, Accad and Calneh in southern Mesopotamia, and then he moved north and built the cities of Ninevah, Reheboth-Ir, Caleh and Resen.
According to Josephus it was under the authority of Nimrod that mankind attempted to build the blasphemous Tower of Babel.
The role of Nimrod in attempting to building a massive skyscraper to heaven makes sense only with the understanding that Nimrod was a worshiper and follower of the chief of the "sons of God" that we know as Satan. The Tower of Babel was an attempt to gain for Satan what Satan desired according to Isaiah 14:13-14,
“You said in your heart, ‘I will ascend to heaven; I will raise my throne above the stars of God; I will sit enthroned on the mount of assembly, on the utmost heights of the sacred mountain. I will ascend above the tops of the clouds; I will make myself like the Most High.’”
Much of humanity was forcefully united under Nimrod, and with him as the authority mankind began to build the rebellious monument known as the Tower of Babel. To remedy the situation God decreed the establishment of different languages and the division of mankind to the seventy angelic powers to manage the affairs of the earth.
Instead of being united against the Creator, humanity would now be divided against the Creator, which would allow for God to continue with His redemptive plan for all humanity, through His own nation Israel.