Reincarnation and atheism are not mutually exclusive as atheism is simply a lack of belief in a deity. Just like a theist can believe in evolution and an atheist not. But it would be rare for an atheist to believe in reincarnation as usually the type of people who are associated with atheism tend to be rational.
Carl Sagan wrote about those children remembering previous life and took them seriously. But I'm not sure I agree with Sagan here. Thorough scientific investigation will debunk reincarnation. I myself investigated reincarnation and haunted places ("guaranteed to find ghosts" according to some friends) during my "buddhist phase". All turned out to be a bunch of nonsense. No such thing as the paranormal, I concluded. Although I could still be wrong..