As verse 20 summarizes, the first goat was for atoning for the Holy Place, the tabernacle of meeting, and the altar. It was on account of the LORD, and because of the uncleanness of the people, that these things had to be cleansed. This goat was not used to add more sin to the altar, but to make an annual purging of the uncleanness. If you would please turn to Hebrews 9, we will see Christ's fulfillment of the first goat:
Verse 7 says that the priest offered (or brought) blood for himself and for the people's sins, and verse 23 underscores that the part of the ceremony involving bringing blood into the Most Holy Place was for purification or cleansing of the objects. This idea is also found in verse 22, which says that according to the law almost all things are purified with blood.
The Levitical priest used animal blood, but our High Priest entered the Holy of Holies in heaven with His own blood. His blood provides an entrance into the presence of the Supreme God. I don't know if you've ever thought about it, but verse 23 says the heavenly things had to be purified. It's as though simply being in the same universe as humanity meant they had to be cleansed. But Christ's blood provides a cleansing so complete that we now can come boldly to the throne of grace, that we may obtain mercy and find grace to help in time of need. His blood appeases the Father, so we can draw near with a true heart in full assurance of faith, having our hearts sprinkled from an evil conscience.