Verse 51, where it says "and it came to pass the selfsame day that the LORD did bring the children of Israel out of the land of Egypt by their armies," is positioned at the end of this paragraph that begins in verse 43. That does not mean that they were circumcised "the selfsame day," but rather that verse is a reference back to verse 41: "And it came to pass at the end of the four hundred and thirty years."
At this point (verse 51) in the story of the events that took place in Egypt, the males had already taken the Passover, and therefore the men were already circumcised. They couldn't have taken the Passover, which occurred at the beginning of chapter 12, unless they had been circumcised. So this section then, from verse 43 through 51, is a reminder of a command given earlier. They didn't get circumcised after Passover and then immediately get up and leave Egypt within the same few hours period.
