The truth is that neither of them actually wanted a child. The truth is that Lily looked at James and asked, "how can we?" and James shrugged, because they couldn't, they shouldn't, not with things being as they were, but as Lily had never been regular and times were, to put it mildly, stressful, it was too late to do anything else. The truth is, they were scared, all of them, and it *wasn't* the time, not for that. For him.

Not that Remus will ever tell Harry this, though, because the truth is, they loved him once he came.