He was dodging the lightning bolt that he thought would come to strike these men down! That is the import of the words they said against him and Aaron.
This verse is linked to the last phrase of the previous verse: "Him you shall hear, according to all you desired. . . ." The Israelites are to listen to the Prophet because, in their fear at Mount Sinai (Exodus 20:18-19), they had begged not to have any further direct contact with God Himself.