Can I change the Difficulty midway?

You can change it from the menu: System > Change Settings > Star Player Settings.