The problem with trying to lie is that, in the face of someone observant enough, even the slightest slip up can put everything into doubt and reveal a lot more than is necessarily wanted. Joker knows this very well. While he might be poisoned to all hell, he doesn't miss that pause, although the realization is a bit slow in coming.

When it does hit, he huffs out a long sigh. "'Remember' is a bit of a strong word," he admits plainly, trying not to show too much himself. Good thing he's so tired that he can't be bothered for the effort of emotions. "Frankly, I figured I was just hallucinatin'."

He is poisoned, and relieving traumas of his own, after all.