I caught one of the thieves who ran away, and he said he was the leader. Said they'd stop attacking if I let him go. I let him go, he says "Suckers!" Next time, if there is one, we will kill him instead.
Found that other map I missed before:
Journal Entry 29 - Map showing "NE Castle"