There's a whole bunch of things he could pick at with regards to that statement (levitating the tiny iron knife wasn't required, if she failed then there were probably a hundred other ways they could take down the Mother of Invention, why not just make the hole bigger), but he's focusing on the big obvious point in his mind: Peridot's memory probably isn't fine.

"I know two people who've gotten soul damage who've had memory problems as a result! If you think your memory is fine, then you must not realize you've forgotten something in the first place."