The Quarter pony is bred to be an American Quarter Horse built on a smaller scale. Breeders focus most on the height and conformation of the breed, and insist that their ponies display Quarter Horse-type characteristics and stand 11.2 and 14.2 hands (46 to 58 inches, 117 to 147 cm) high. Depending on the registry, the Quarter pony may come in any color or combination or colors, including pinto patterns such as tobiano and overo and spotted Appaloosa patterns. In the early years of the breed, only solid colors were allowed. The breed averages 13.2 hands (54 inches, 137 cm) high, however, some breeders are working to breed taller animals between 13.2 and 14 hands (54 to 56 inches, 137 to 142 cm) high. The breed has a short, broad head with small ears and wide-set eyes, set on a slightly arched neck. The shoulders are sloping, the withers sharp, the chest broad and deep. The back is short and the hindquarters broad and deep.

Quarter ponies are often used in western riding activities as mounts for children because of their small size, and calm, even temperament. Larger ponies are more suitable for adult riders and sometimes used for rodeo events such as steer wrestling.

The Quarter pony is recognized by several different breed registries that each have different requirements. The American Quarter Pony Association requires that, although parentage may be unknown, the pony must have conformation that is desirable for breeding and be easily recognizable as having Quarter pony or Quarter Horse breeding. Pinto, Appaloosa and white horses are not eligible for registration, nor are gaited ponies. The National Quarter Pony Association requires that stallions be registered with the AQHA before they can be registered with the NQPA. Mares must have one parent registered with the AQHA, be registered with the AQHA themselves, or go through a special registration process. Geldings simply have to be of Quarter horse type to be eligible for registration. Horses with pinto or Appaloosa markings, or with excessive white, are not eligible for registration. The International Quarter Pony Association allows pinto and Appaloosa markings, and simply requires that ponies be of Quarter-type conformation and good disposition for registry. Any type of pony meeting these requirements may be registered through the hardship registration program, which includes a special inspection. However, if ponies have a parent registered with an approved breed registry (approved breeds include the Quarter pony, Quarter Horse, Paint horse, Appaloosa and Pony of the Americas), they are automatically eligible for registration, with no inspection required. Crosses with gaited breeds are not accepted for registration.