These verses clearly show that the leaving by night pertained to the first Day of Unleavened Bread—the 15th, not the 14th.
Passover is the 14th, and the starting point. According to verses 2 and 3, it was in Rameses, not their houses. It was the night part of the 15th day of the month that they left Egypt. It was the 15th day of the first month, on the day after the Passover.
The Egyptians would not have been burying their dead just minutes after they died on the 14th in the dead of night. They were burying their dead late on the daylight portion of the 14th toward sunset, as the Israelites were beginning to move out of Rameses. That is four differences already.
Now why would they celebrate all night? They were commemorating coming out of Egypt, and in all likelihood the Israelites, once they began to leave Rameses, did not stop till daylight the next day. They marched all night, and then rested in the heat of the day. They were celebrating "The Night To Be Much Observed"—a night of watching. God was watching over them as they left Egypt by night. Now verse 8 makes sense as to why He would have to tell them to leave off the celebration and go back to their tents.
This passage is all about the First Day of Unleavened Bread, beginning with the night of solemn observance, just after sunset. Unleavened Bread pictures God's deliverance from the environment of sin. It is about what God did and does for us - it is a memorial of His deliverance. But notice where this took place: Verse 37 puts the beginning of the exodus at Succoth. That was the first place they camped after leaving Ramses in Egypt proper.
So, this is a commonality with Tabernacles, because the place the Israelites camped for the First Day of Unleavened Bread is what this feast is named after. The Jews call this Feast Sukkot, which is the Hebrew word for tabernacle, tent, or booth. And the name of the first place the Israelites camped is Succoth, which is the same word, except that it is plural. So, the Israelites stayed in booths in Succoth. This is curious because they stayed in booths in the place that had already been named booths. It was already named Succoth when they got there and made their camp. The backstory to this is found in Genesis 33:17:
So, the place the Israelites camped for the First Day of Unleavened Bread was named after the dwellings Jacob made for his livestock. Therefore the name of the place is called Succoth.
Notice the language in Exodus 12:17, again. "For on this same day I will have brought your armies out of the land of Egypt." Look again at another place:
He's talking about the fifteenth!
So we see the same language here. They went out on the fifteenth. And verse 22, of this chapter, proves that none of these verses (17, 41, or 51) could refer to Passover day because (in verse 22) they had to stay in their houses until morning. They were still in their houses when daylight of the fourteenth came, and they had not even gone to Rameses yet.
They went out, but they did not go out in a disorganized way. Everybody was not just tramping, tramping, tramping—running things over, or whatever. They were all lined up. And, because of Scriptures like this, I do not believe that Moses merely assembled everybody in one huge mass, like a huge crowd attending a rock concert, in some open area of Rameses. It would not have worked that way. The indication, from these Scriptures, is that he had them lined up as in a parade—according to their tribes, according to their families. It was not a milling mass of people, just running in every direction. Rather, he had them lined up—like a parade.
So we see the same language here. They went out on the fifteenth. And verse 22, of this chapter, proves that none of these verses (17, 41, or 51) could refer to Passover day because (in verse 22) they had to stay in their houses until morning. They were still in their houses when daylight of the fourteenth came, and they had not even gone to Rameses yet.
They went out, but they did not go out in a disorganized way. Everybody was not just tramping, tramping, tramping—running things over, or whatever. They were all lined up. And, because of Scriptures like this, I do not believe that Moses merely assembled everybody in one huge mass, like a huge crowd attending a rock concert, in some open area of Rameses. It would not have worked that way. The indication, from these Scriptures, is that he had them lined up as in a parade—according to their tribes, according to their families. It was not a milling mass of people, just running in every direction. Rather, he had them lined up—like a parade.