or just switched in terms of which hip he wore his scabbard

If that is the Tower of Joy, and that IS Ned... Somewhere along Ned’s life he switched from a lefty to a righty