"Mn, I see!" No he doesn't. But he's absolutely walking with the other, no need for dragging. "I, the man Gongenzaka, had merely intended to return to my own room- so I do not indeed have anywhere else to be!"

So it's fine to go to yours, in other words.