Horses were originally meant to be purchased through an in-game menu like other items in the game. In the final game, they are purchase through dialogue options. There exist unused lines for these purchases, "Need a new mount?" and "Looking to buy a horse? I have some fine animals for sale."
Contributed by ClaudX