A little leaven leavens the whole lump.
— Galatians 5:9
Israel committed a great sin. They knew better. They were God's chosen, covenant people. They had seen his power demonstrated time and time again. They saw the Red Sea parted as they passed through, and they saw it close on the pursuing Egyptian army. They saw manna provided every morning. They saw God's fire by night and His cloud by day. They saw miracle after miracle. They made a promise to obey God on three separate occasions. Much had been given to Israel, and much was expected from Israel.
So when they worshiped the golden calf that Aaron formed from the jewelry they willingly gave him, it was a radical sin. And it would be dealt with harshly. When Moses arrived, he threw down the commandments, took their golden calf, ground it into powder, put it into water, and then made them drink it. Then he gave the command for a number of them to be put to death for their sin.
It almost seems unfair that God would deal so harshly with these people. But frankly, God doesn't owe us the time of day, much less an explanation of why He does or does not do certain things. He just does what He is going to do.
Basically God was saying, "I hate sin. It will not be tolerated. It must be dealt with swiftly, lest it spreads and do even more harm."
The Bible compares sin to leaven, which is yeast that is put into bread to cause it to rise before baking. It is always a picture of evil in the Bible. We could say that sin is like cancer. It needs to be cut out before it metastasizes, before it spreads through someone's system. That is why the Lord tells us to deal quickly with sin.