While wearing the glove, which part of the glove is not sterile? I have a hard time understanding this.

Glove pairs are packaged in individual sterile wrappers. Gloves are packaged so that they can be worn without contaminating the glove’s outer surfaces.  It is important that the hands are kept above the  waist at all times in order to avoid sterile glove contamination. In addition, the hands are not sterile, and so care should be taken not touch the outside of sterile gloves with bare hands. For an illustrated 13 step process on the proper way to don sterile gloves, kindly visit this link.

Ref.