Here in Michigan the oldest grave markers that you will find average to have been erected around 1850. Before that, there was typically only markers made of wood.
A persons immediate family would be able to visit the grave of a loved one as long as they lived in the area. After that, it was neglected and forgotten. Eventually giving in to time and weather and fading along with the memory of the person who died. The only persons with a more permanent marker would have been dignitaries or important people of the area or time. But, often they were memorialized in a town square rather than a grave yard. Grave yards were not how we know them today. They were often outside of town because the art of embalming had not been developed and lets face it, a decaying corpse is not something that anyone would want to smell. Today, we not only have embalming, but we have cement vaults to contain our loved ones. They not only help to keep out moisture, bugs and tree roots, but they contain the not so pleasant side of death. We would all like to remember our loved ones for who they were, alive and not think about what's happening beneath the surface of the cemetery plot.