In playing with my friend we noticed that our experience points were out of sync. After replaying through we discovered that the Imprisoned Elf quest line is not awarding party wide experience.
Had the same issue, very annoying. Reloaded multiple times, everyone had the "share journal" button toggled, but no matter what, whoever turns in the quests gets the exp.