JOEL - Speaking about the Nauvoo Temple to be built God said:

43 And ye shall build it on the place where you have contemplated building it, for that is the spot which I have chosen for you to build it.
44 If ye labor with all your might, I will consecrate that spot that it shall be made holy.
45 And if my people will hearken unto my voice, and unto the voice of my servants whom I have appointed to lead my people, behold, verily I say unto you, they shall not be moved out of their place.
46 But if they will not hearken to my voice, nor unto the voice of these men whom I have appointed, they shall not be lest, because they pollute mine holy grounds, and mine holy ordinances, and charters, and my holy words which I give unto them. (D&C 124: 43-46)

This revelation was given to all the saints in 1841, three years before Joseph Smith died. After Joseph died, there was much dissension among the members about who should be the new leader of the Church.
Following instructions already given in the Doctrine and Covenants (D&C 107), the Twelve Apostles assumed leadership as a group, which was supported by the majority of members. But there were many who didn't agree with this and therefore didn't hearken to God's voice or the voice of their new leaders as they were told to do 3 years ago. As a result there were a few former church leaders who left and and took many of the saints away from the main body. Had they all stayed together in unity they may have been able to better withstand the persecution from enemies of the Church as God promised. So it wasn't Brigham Young's fault; it was the fault of those who did not hearken to the voice of God's leaders as they were told to do in verse 45, and so because of them the main body of members had to "move out of their place" because of the persecution that followed, some of which had the support of those former Church members.