Stephen Dorff almost quit acting after his brother’s death