Yōga Sūtra 1.15

दृष्टानुश्रविकविषयवितृष्णस्य वशीकारसञ्ज्ञा वैराग्यम्।

Dispassion (vairāgya) is withdrawing from the craving of topics that are experienced, grasped through sensual percept of the scene, and heard of.

While worldly pursuit requires passion, exactly the opposite, dispassion, is required for pursuit of Yōga. Developing disinterest in worldly topics is the most difficult aspect of Yōga. Inspite of all physical exercises, breathing exercises etc., if one does not develop dispassion, attainment of end goal of Yōga remains elusive.