You are probably in sorted mode (alphabetical, by date, etc). Set Priority Matrix to manual order mode and try again.

See the video below for a visual explanation for how to move items on most of the available platforms.