They could, and they barely refrain from pointing that out, because he knows they probably wouldn't. They don't want to ramp up old anxieties about the Lord of Shades.

Elderbug worries. I think that's somewhat sensible of someone in his place. Was Dirtmouth just him when you arrived as well? While they're trying to move away from their own concern, the curiosity behind the question isn't false. They don't know how long Lemm's been here. With the strange stasis keeping the likes of Hornet and Nailsmith alive (setting aside Quirrel), it could be far longer than they've previously thought.