Obama portrayed black people as having an inner white person, and he wondered "whether the bonds of community could be restored without collectively exorcising that ghostly figure that haunted black dreams." So that inner white person wasn't real. It was a ghost — a ghost that haunted dreams. The book is "Dreams From My Father," so it's quite significant to find the notion of dreams haunted by white people, and white people conceptualized as ghosts.
She found 8 different examples of ghosts referred to in the book... very interesting stuff, go read the whole thing.