These verses clearly show that the leaving by night pertained to the first Day of Unleavened Bread—the 15th, not the 14th.
Passover is the 14th, and the starting point. According to verses 2 and 3, it was in Rameses, not their houses. It was the night part of the 15th day of the month that they left Egypt. It was the 15th day of the first month, on the day after the Passover.
The Egyptians would not have been burying their dead just minutes after they died on the 14th in the dead of night. They were burying their dead late on the daylight portion of the 14th toward sunset, as the Israelites were beginning to move out of Rameses. That is four differences already.
Now why would they celebrate all night? They were commemorating coming out of Egypt, and in all likelihood the Israelites, once they began to leave Rameses, did not stop till daylight the next day. They marched all night, and then rested in the heat of the day. They were celebrating "The Night To Be Much Observed"—a night of watching. God was watching over them as they left Egypt by night. Now verse 8 makes sense as to why He would have to tell them to leave off the celebration and go back to their tents.
This passage is all about the First Day of Unleavened Bread, beginning with the night of solemn observance, just after sunset. Unleavened Bread pictures God's deliverance from the environment of sin. It is about what God did and does for us - it is a memorial of His deliverance. But notice where this took place: Verse 37 puts the beginning of the exodus at Succoth. That was the first place they camped after leaving Ramses in Egypt proper.
So, this is a commonality with Tabernacles, because the place the Israelites camped for the First Day of Unleavened Bread is what this feast is named after. The Jews call this Feast Sukkot, which is the Hebrew word for tabernacle, tent, or booth. And the name of the first place the Israelites camped is Succoth, which is the same word, except that it is plural. So, the Israelites stayed in booths in Succoth. This is curious because they stayed in booths in the place that had already been named booths. It was already named Succoth when they got there and made their camp. The backstory to this is found in Genesis 33:17:
So, the place the Israelites camped for the First Day of Unleavened Bread was named after the dwellings Jacob made for his livestock. Therefore the name of the place is called Succoth.
So there is also a solemn aspect to all of this, because of what had been wrought in Egypt—what had been necessary to redeem them, to free them, and to bring them out. But it is God's actions on this particular night that I want to highlight. Early in our conversion, we are very aware (in our first love) of God watching. That is what it basically says here. We observe this night because God observed His people going out of Egypt. You see—God was there. He was involved. He was so involved that it says (back in Exodus 11:7), when He was telling Moses what was going to happen, He said, "Not even a dog will wag his tongue against you." (Meaning, not even a dog will bark as you leave Egypt.) That is how closely God supervised their leaving.
Notice, rather than leaving all physical possessions behind like those pioneers needed to do to lighten their load, God provided much more physically for their journey. They added great material possessions from the spoils of Egypt before their journey out began with setting sun that started the 15th day, and the first day of Unleavened Bread.
Notice it says in verse 35 that they had done all this at the word of Moses.
This was 40 years to the selfsame day! Why is that used? "Selfsame day" is used to draw attention in Exodus 12:40-42 to the Night To Be Much Observed - the night that memorializes God's faithfulness to His covenant with Abraham, and Christ's sacrifice and entombment for our sins, and thus Israel's release from their physical bondage, and our release from our spiritual bondage to this world.
