Nobles from the Empire will demand all the privileges of their title. Your own titled colonists will be more flexible, unless they are greedy or jealous.