The brothers were afraid they would be arrested and taken as slaves because of the money left in their bags on the first journey.