As verse 20 summarizes, the first goat was for atoning for the Holy Place, the tabernacle of meeting, and the altar. It was on account of the LORD, and because of the uncleanness of the people, that these things had to be cleansed. This goat was not used to add more sin to the altar, but to make an annual purging of the uncleanness. If you would please turn to Hebrews 9, we will see Christ's fulfillment of the first goat:
Verse 7 says that the priest offered (or brought) blood for himself and for the people's sins, and verse 23 underscores that the part of the ceremony involving bringing blood into the Most Holy Place was for purification or cleansing of the objects. This idea is also found in verse 22, which says that according to the law almost all things are purified with blood.
The Levitical priest used animal blood, but our High Priest entered the Holy of Holies in heaven with His own blood. His blood provides an entrance into the presence of the Supreme God. I don't know if you've ever thought about it, but verse 23 says the heavenly things had to be purified. It's as though simply being in the same universe as humanity meant they had to be cleansed. But Christ's blood provides a cleansing so complete that we now can come boldly to the throne of grace, that we may obtain mercy and find grace to help in time of need. His blood appeases the Father, so we can draw near with a true heart in full assurance of faith, having our hearts sprinkled from an evil conscience.
There is not time to explore this completely, but notice that Jesus says that the Passover will not be fulfilled until the Kingdom of God. It is easy to read right over that, but it is a critical part of Passover. The sin offering has already been fulfilled (Hebrews 1:3; 7:27; 9:12-14, 26-28; 10:12, 18), but the Passover has not, as it says here. This is because of Passover's theme of redemption, which is actually a process. This is why Scripture speaks of redemption in both the past (Romans 3:23-25; Ephesians 1:7; Colossians 1:14) and future tenses (Luke 21:28; Romans 8:23-25; Ephesians 1:13-14; 4:30). We have been redeemed from the death penalty, but we still need to be redeemed from this corrupted nature that incurs the death penalty (see Titus 2:11-14). God must redeem us from this flesh - this body of death, as Paul calls it - so we can fully manifest the image of our Creator. Thus, our final redemption occurs when we have been resurrected and have fully taken on the incorruptible nature (see I Corinthians 15:50). At that point, Passover will be fulfilled.