He will be provided with a horse, but it's a piece of work that needs to be tamed first. He can find it in Dayrise, where some of the law enforcement horse stables are (along with Dayrise carriages, if he wants to give them a try).