What started it for me was the Book of Abraham. I sat and pondered the facsimiles (especially number 3 and whether it was a scene that could represent Abraham on Pharaoh's throne or something else) and wondered whether or not the symbols on the hieroglyphs had ever been actually translated to vindicate Joseph's claim. When I read up on the question, it turns out they had and Joseph Smith was wrong. The papyrus scrolls contained nothing more than funerary prayers for the departed. It was then that I began to ask the question, If Joseph was wrong in his translation of some Egyptian papyrus, what else could he have been wrong about? This opened everything up for me.
I read and grappled with all the apologetic responses, but they all came out sounding like excuses to me. I also discovered a pattern in the method of LDS apologists. The responses I found tended to fall into the pattern of 1) Deny the questions' relevance 2) Deflect attention away from the question at hand and if those two methods don't work 3) Overwhelm with information. This, of course, called into question all of Joseph's ability to translate for me.
Once I came to the realization that Joseph probably did not have all the answers, I began to see things throughout the scriptures that didn't add up. I am amazed at how much I did not see before. For example D&C 132:54 tells Emma that she will be destroyed if she doesn't go along with Joseph's polygamy. Why did God feel the need to threaten her with destruction? Then there is the story surrounding D&C 111. Basically Joseph was told there was treasure to be obtained in Salem, MA. He went there on a treasure hunt and found nothing. Did God lie to Joseph or was he just mistaken? There are many others that began to stick out as glaring examples of outright deception at the worst to confusion about what God meant at the best. I concluded (from these and many, many other examples from church history) that maybe Joseph wasn't a prophet at all, but merely a man who was either actively deceiving everyone or truly believed in his calling and was just doing what he thought was best. It became apparent to me that regardless of his intent (both conclusions can be well supported) he wasn't anything spectacular in the overall scheme of history as far as what he claimed compared to others in his day. In fact, I believe if it weren't for the saints' migration to Utah, the church would probably be an insignificant movement today.
My conclusions didn't happen overnight. It took me literally a solid year of reading and studying different things until I finally drew the conclusions that I did. I was a serious student of Joseph Smith and the origins of Mormonism. I read Fawn Brodie's No Man Knows My History, I perused Robert Bushman's Rough Stone Rolling, I read Charles Larson's book, By His Own Hand Upon Papyrus, I read the different accounts of Josephs first vision, I read a biography of Sidney Rigdon, I read Michael D. Quinn's Mormonism and the magic world view, I read up on the Solomon Spaulding controversy, I read from the Journal of Discourses, etc., etc.
Then one night, about a year and a half ago, in a hotel room in Kansas City Missouri I knelt and prayed for a truly objective answer to prayer. The only answer I have ever received since that day is silence. God has apparently forsaken me in my desire to obtain a faithful explanation for all the questions and concerns I had.
Now things jump out at me and I am continually facing new realizations about how things just don't make sense about what I used to believe. Things that are said in church really bother me. Things in the scriptures bother me. When you say, "Like what?" I could do a commentary on all the standard works and point out example after example of things that just don't add up for me.
I am at a point now, where I am struggling to re-orient myself and re-establish a basis for my moral code. So far the best I have come up with is that life and it's preservation is of utmost importance to me. I can't explain or even begin to guess what will happen in the hereafter, but I don't worry about it anymore (and that is such a relief!).
I feel truly free to live my life as I desire, no longer in fear of what the church thinks or wants me to be doing. I have taken back my power and can say no to callings, etc. One thing I remember saying when I got to the point I am at is "Thankfully, now I can get off the merry-go-round". That was what the church had become for me, a merry-go-round that just never stopped. I used to think of the body of the church as a train, that I had to make sure I stayed on so I didn't get left behind. Now I realize that, if it is a train, it is moving in a small circle. Never arriving, just coming around again and again and rather than trying to get aboard and make sure I stay on, it took getting off to have things make sense again.