Are all 4 character portraits linked together? If any of the portraits are separate, click and drag them together (in the controller UI, hold the left trigger, and use the A, Y and left stick to adjust groups).
If you select and move Roderick, does the rest of the group follow?