That's due to changes in air pressure. Air exerts more pressure at lower altitudes than higher ones. Even though you dont see it, air has a weight to it. Think about the column of air directly overhead of you. As you go higher and higher in altitude, there's less air above you, so the cumulative "weight" of air over your body decreases, translating into a drop in air pressure.
You'll notice the quick drop in pressures because the air inside your body will then be higher than the pressure outside your body. When your ears "pop", that's the body's way of alleviating the pressure difference between the air outside your body, and within.