"But if he knew something was wrong, he shouldn't have been trying to do reckless stuff like drinking with you anyway! That's not making sure you're okay."

That's where Yuya's concern is. Not in the result, but in the intentions that caused it to begin with. "He's really careless with you... does that not bother you?"