The more fun you want to have on your motorcycle, the more important fitness becomes. A good level of motorcycle fitness can help with stamina, endurance, and of course lifting your bike in the event that it goes down.
That's particularly helpful if you do a lot of dual sport or adventure riding, where you reasonably expect that you're going to have to lift your bike back up at one point or another. But even if you plan to stay on road at all times, you can't always predict when you're going to need to pick up your (or someone else's) bike after an off. Stuff happens, and it's usually not convenient when it does.