Is there a being on this earth that doesn't appreciate praise? I don't think so. I know I like to receive attention for a job well done. It certainly looks like my dogs do. And, to a certain extent, I believe horses understand praise too. I do not believe that horses have any inherent wish to please us, so that makes it doubly important to understand the difference between treats and rewards, and that rewarding a horse effectively takes some skill.
The key, I believe to praising and rewarding horses is immediacy and consistency. You and I might recognize that if we work hard every day, we'll get money at the end of the week. Horses don't understand 'in an hour', 'in a few days' or 'next Friday'. They live in the present and won't understand that the tub of carrots you fed them in the evening was because they won a ribbon at the horse show that morning. To them, they probably had a possibly unpleasant day, followed by an unexpected treat. It has no more meaning to them than if you walked out to the pasture at random times and handed them a carrot. So it's important to understand the difference between a treat and a reward. Treats come at random times, for no apparent reason. Praise or reward come immediately after a desired behavior by the horse.
Pats and Scratches
There are a few different ways you can praise or reward your horse. Often, while we are handling our horses or riding, we'll give an affectionate pat immediately after the horse behaves in a certain way. For example, if you are teaching your horse to do a rein back, you may pat or scratch your horse's neck each time they take a calm step backwards. Some people are 'scratchers', and some are 'patters'. I tend to be a scratcher, because I know the spot along Trillium's neck that she likes to be scratched and feel it's a more effective form of praise than a pat—which could also feel like you're swatting away mosquitoes. A calm, happy voice can accompany your pat or scratch. This, however, may be the least effective form of praise.
As you ride, the quieting of your hand, seat and leg aids is also a form of reward. Your horse will come to know that each time he performs a desired maneuver he is rewarded and told 'yes, you're doing it right' because your touch becomes lighter as he carries himself or moves as you ask. For example, if you're asking your horse to be more forward as he trots, you'll cue with your legs and seat to encourage a brisker trot, but soften your aids when the tempo and speed you want are reached. If the horse begins to slacken, you'll step up your aids again. He'll learn that the reward for doing exactly what you want, is that he won't have to put up with your insistence. I spent a lot of time teaching Trillium and myself to do a leg yield. I would cue and as soon as she offered to bend and move sideways, I would soften the cue (while the Halleluiah Chorus went off in my head if we got more than one or two steps).
Another type of praise or reward are edible treats. Some people would argue that food treats have no place in training your horse. I disagree. I think conscientiously done, food rewards can be very effective. As I mentioned above, a treat long after the desired behavior has passed is nothing more than a random snack. However, if treats are immediately and consistently given following a desired behavior, they can be very effective. Anyone who has clicker trained a horse will know that many horses (but not all) are very motivated to earn food treats, and training goes quickly once the horse understands something tasty is the reward for a job well done. It is possible to use food rewards from the saddle. Using a clicker to pinpoint the correct behavior and bridge the time to the delivery of the treat may be the best way to do this.
It is just as important to make sure your horse understands that it must be polite when taking a reward, just as I describe in Feeding Horses Treats. It's important, whether just feeding treats or doling out rewards, that our horses don't see us as slot machines, into which they can put a coin, and out pops random and unexplained wealth or nothing at all. Rewarding effectively is a skill you must learn for it to be most effective.