Board Thread:The Panopticon/@comment-5940646-20151203081819/@comment-5918438-20151226182323

Logically, it doesn't make sense that the Doctor's brain could hold memories of a life he'd never personally lived. But it's not what logical that counts. It's what the story says. I'm not sure that line about remembering is unambiguously about remembering the previous loops, but that seems to be the most obvious interpretation, given the timing. He says it at the time where, at least in other loops, he makes the speech about the bird and realises what's been going on.