Joseph's brothers were worried that Joseph would repay them for all the evil the brothers had done to Joseph.