That's the problem: he doesn't want to talk. All he wants to do is hide somewhere dark and suffocating, let it drown everything else until he can put on the so reliable mask he's kept on before, and act like this never happened. Like Snake doesn't (rightly) blame him for what happened, like he hasn't broken down.

He just wants to sweep it all under the rug.

But looking at Psi, tired and wanting to help.... It's hard to refuse the offer to at least go with him. So, exhausted himself, Joker just gives a limp nod.

Maybe by the time they get up there... he'll be able to fake it all again.