Science Dawg

Effectiveness of Aversive v Reward Based Training

Introduction

 Dog training is essential if desired behaviours are to be displayed and a strong owner-dog bond established. Surveys indicate that up to 90% of dogs display problem behaviours, with these behaviours being the major reason for relinquishment and euthanasia (Yin et al., 2008). Therefore, it is essential that techniques and training programs are established to effectively extinguish these behaviours. Traditional methods incorporated aversive and often confrontational techniques, which often have negative consequences (Haverbeke et al., 2008). Recent studies have suggested that reward-based programs, such as using clicker devices and food, are more effective and offer improved welfare for animals (Herron et al., 2009).

Discussion

 Traditional aversive techniques and positive punishment often result in an increased arousal and heightened fear of the owner, perhaps provoking defensive aggression and putting owners at risk of injury (Herron et al., 2009). Dogs can communicate with trainers but they are more likely to ignore trainer commands if persistent aversive correction is used. This method may also lead to a situation of “learned helplessness” (Bentosela et al., 2008). Mild aversive methods may momentarily disrupt undesirable behaviours, but do not selectively reinforce desirable behaviours and are inefficient in rectifying problem behaviours (Herron et al., 2009). Furthermore, they may perpetuate underlying anxiety problems, compromising welfare and increasing the occurrence of defensive aggression, perhaps teaching the dog to bite without warning (Herron et al., 2009; Landsberg et al., 2003). Compromised welfare is often reflected in a lowered body posture and exhibition of defensive aggression to perceived threats (Haverbeke et al., 2008; Herron et al., 2009).

In their investigations, Haverbeke et al. (2008) and Herron et al. (2009) examined the effects of aversive training methods on dogs. Herron et al. (2009) listed 30 possible interventions used to deter or promote certain behaviours and asked 140 interviewees, who had responded to the survey and reported behavioural problems in their dogs, to record the responses of their dogs. Interventions were recorded as “aversive: direct confrontation”, “aversive: indirect confrontation”, “reward training” and “neutral”. Interviewees then recorded whether the technique used induced “positive”, “negative” or “no effect” on behaviour, and whether or not it elicited a “growl/bare teeth”, “snap/lunge”, or “bite” from the dog. In addition, dog owners were also surveyed about the effectiveness of positive reinforcement interventions.

Common methods used to “correct” behaviour included leash correction (75% of interviewees), hitting or kicking the dog (43%), “check chains” and muzzles (both 38%). Similarly, indirect confrontational methods such as “staring down” or growling at dogs also elicited aggressive responses. Staring down and growling prompted aggressive responses from at least a quarter of dogs on which they were attempted. Furthermore, certain methods of correction were more likely to elicit aggressive responses in certain dogs. Examples include the “alpha roll”, where the dog was forced onto its back and held there, and yelling “no” at dogs presenting for aggression towards familiar people.

Similarly, Haverbeke et al. (2008) recorded and reviewed two training sessions by 33 Belgian military dog-handling teams. Handlers evaluated dogs via a series of eight obedience exercises and five protection exercises, punishing or rewarding dogs at their discretion. Observed parameters were team performance, handler’s behaviour and dog’s behaviour.

Techniques used by dog handlers were primarily aversive. Leash correction and hanging dogs by their collars were the primary means of behaviour correction. Once again, these methods incited aggressive responses, with one dog biting its trainer. Moreover, they had an obvious effect on welfare, with dogs commonly exposed to punishment exhibiting much lower body postures.

In contrast, non-aversive, reward-based interventions rarely elicit aggressive responses (Haverbeke et al., 2008; Herron et al., 2009) and are significantly more successful in promoting desired behaviours (Yin et al., 2008). This was particularly evident in the investigation by Haverbeke et al. (2008), when four trainers opting for a more reward-based approach had considerably more success (finishing first, third, fifth and ninth out of the 33 teams) than those primarily using punishment.

Rewards-based training and positive reinforcement, such as “clicker” training, to induce favourable behaviours, have increased in popularity in recent times (Smith & Davis, 2008). These techniques are more effective and offer higher levels of obedience than programs using aversive interventions, particularly punishment (Herron et al., 2009; Hiby et al., 2004). Furthermore, dogs are less likely to develop future problems such as fear-related responses (Blackwell et al., 2007) and ultimately are less stressed, subjected to less pain, and therefore less likely to injure their owners (Herron et al., 2009).

In their trial, Smith & Davis (2008) investigated the efficacy of clicker training in pet dogs. Eighteen Basenjis were conditioned to associate the “click” (secondary reinforcer) with food (primary reinforcer) by delivering food one second after the “click”. The control group received food alone after one second. Dogs were then trained to nose-touch an orange traffic cone. Efficacy was measured by success in performing the behaviour, time taken for training and extinction of the behaviour. Results suggested that while clicker training may not “speed up” training, it does lead to increased resistance to extinction (Smith & Davis, 2008).

Clicker training is beneficial in circumstances where it is difficult or impractical to reward an animal immediately after it has performed a desirable behaviour, such as when dogs are a considerable distance away from the trainer (Smith & Davis, 2008). Initially associating the clicker with food increases the behaviour’s resistance to extinction, despite the absence of a primary reinforcer. This is particularly significant as it offers an alternative to some more aversive, confrontational methods commonly used in long-distance training.

Conclusion

Recently, the effectiveness of traditional, aversive training methods for dogs has been questioned. Aversive techniques are often more difficult to use correctly and so abuse or accidental misuse by inexperienced trainers may seriously compromise animal welfare and increase the risk of trainer injury. Consequently, alternatives have been sought to address these issues. Reward-based training regimes have increased in popularity, offering improved efficacy and improved animal welfare. Further research into the advantages of reward-based programs is necessary to improve the efficacy of these techniques and further strengthen the owner-dog bond.

References

Bentosela, M., Barrera, G., Jackovcevic, A., Elgier, A.M., Mustaca, A.E. (2008) Effect of reinforcement, reinforcer omission and extinction on a communicative response in domestic animals (Canis familiaris). Behavioural Processes 788, 464-469.

Blackwell, E.J., Twells, C., Seawright, A., Casey, R.A. (2007) The relationship between training methods and the occurrence of behaviour problems in a population of domestic dogs. Proceedings of the 6th International Veterinary Behaviour Meeting, Fondazione Iniziative Zooprofilattiche e Zootecniche, Brescia, Italy, 51-52.

Haverbeke, A., Laporte, B., Depeierieux, E., Giffroy, J.M., Diedrich, C. (2008) Training methods of military dog handlers and their effects on team’s performances. Applied Animal Behaviour Science 113, 110-122.

Herron, M.E., Shofer, F.S., Reisner, I.R. (2009) Survey of the use and outcome of confrontational and non-confrontational training methods in client owned dogs showing undesired behaviours. Applied Animal Behaviour Science 117, 47-54.

Hiby, E.F., Rooney, N.J., Bradshaw, J.W.S. (2004) Dog training methods – their use, effectiveness and interaction with behaviour and welfare. Animal Welfare 13, 63-69.

Landsberg, G.M., Hunthausen, W., Ackerman, L. (2003) Canine aggression, Handbook of Behaviour Problems of the Dog and Cat (second ed.), Saunders, Edinburgh, 385-426.

Smith S.M., Davis, E.S. (2008) Clicker increases resistance to extinction but does not decrease training time of a simple operant task in domestic dogs (Canis familiaris). Applied Animal Behaviour Science 110, 318-329.

Yin, S., Fernandez, E.J., Pagan, S., Richardson, S.L., Snyder, G. (2008) Efficacy of a remote controlled, positive reinforcement, dog-training system for modifying problem behaviours exhibited when people arrive at the door. Applied Animal Behaviour Science 113, 123-128.

Fairy Tales: Top 10 Dog Behaviour Myths

The Need to Feed: Studies indicate dopamine plays a role in food reward

Abstract

The ability of food to establish and maintain response habits and conditioned preferences depends largely on the function of brain dopamine systems. While dopaminergic transmission in the nucleus accumbens appears sufficient for some forms of reward, the role of dopamine in food reward does not appear to be restricted to this region. Dopamine plays an important role in both the ability to energize feeding and to reinforce food-seeking behaviour; the role in energizing feeding is secondary to the prerequisite role in reinforcement. Dopaminergic activation is triggered by the auditory and visual as well as the tactile, olfactory, and gustatory stimuli of foods. While dopamine plays a central role in the feeding and food-seeking of normal animals, some food rewarded learning can be seen in genetically engineered dopamine-deficient mice.

Introduction

While the cognitive and behavioural disturbance we call ‘hunger’ is innate, the appetites for specific foods are learned. Undifferiented hunger is controlled largely by fluctuations of peripheral and hypothalamic peptides (Saper et al. 2002; Horvath & Diano 2004) and thirst is controlled by fluctuation in vagal input (Kraly et al. 1975) triggered by hypovolemia (Fitzsimons 1961) and by dehydration of cells in the lateral preoptic area of the hypothalamus (Blass & Epstein 1971; Peck & Novin 1971), However, neither hunger nor thirst results in unconditioned goal-directed behaviour (Changizi et al. 2002). Chance encounters with the sweet (Pfaffmann 1960) or salty (Denton 1982) taste of preferred foods or with the oral cooling by ingested fluids (Mendelson & Chillag 1970; Freed & Mendelson 1974) are required before goal-directed behaviour results from the interaction of internal need states with the salience of environmental cues (Bindra 1972). The infant recognizes (Steiner et al. 2001) and can learn to seek out (Johanson & Hall 1979) sweet tastants, but the appetite for a specific food is controlled by the interaction of the hunger-associated peptide levels with the brain circuitry that codes the animal's reinforcement history with that food. Until it has received reinforcing feedback from various foods, the infant indiscriminately mouths both food and non-food objects. The monkey's appetite for a yellow banana depends on the prior learning of the association of the sight of the yellow banana skin with the sweet taste of the white banana meat (Wise 2004b) and with the post-ingestive consequences of the ingested fruit (Le Magnen 1959). Similarly, the vitamin-deficient rat does not innately know what foods contain the deficient vitamin. Rather, the vitamin-deficient rat progressively loses its food neophobia until, by sampling new foods randomly, it chances on and ingests a food with the missing vitamin (Rodgers & Rozin 1966; Rozin & Rodgers 1967; Rozin 1969). The specific preference for a particular substance is only established when the post-ingestional consequences of the repleting food stamp in or ‘reinforce’ the tendency to approach that food (Rozin & Kalat 1971). Similarly, food- or water-seeking behaviours develop only after the animal has had the paired experience of hunger with eating or thirst with drinking, respectively (Changizi et al. 2002).

Our understanding of the brain circuitry through which various rewards control behaviour began with the findings that rats would learn to work for the direct electrical stimulation of the brain (Olds & Milner 1954) or for the pharmacological stimulation of the brain by psychomotor stimulant drugs (Pickens & Harris 1968). The finding that the rewarding effects of brain stimulation (Liebman & Butcher 1974; Fouriezos & Wise 1976) and of psychomotor stimulants (Yokel & Wise 1975; de Wit & Wise 1977) was blocked or attenuated by dopamine antagonists first implicated brain dopamine in reward function. Similar attenuation of food reward by dopamine antagonists (Wise et al. 1978a,b) first implicated brain dopamine in the control of behaviour by natural rewards.

Importance of dopamine for non-food reinforcement

Dopamine antagonists impair learning (Wise & Schwartz 1981) and, by extinguishing them, previously learned (Wise et al. 1978a,b) instrumental responding for food. Several lines of study confirm that they do so by blunting reward function itself (Wise 1982, 2004a; Beninger 1983; Smith 1995) rather than, as has been suggested (Mason et al. 1980; Koob 1982; Tombaugh et al. 1982; Salamone 1986), by simply impairing performance capacity.

The earliest evidence that dopamine plays an important role in motivational function was that brain stimulation and psychomotor stimulants were simply ineffective as reinforcers in animals treated with response-sparing doses of dopamine antagonists. Intravenous amphetamine and cocaine failed to maintain responding when tested under the influence of dopamine antagonists, despite evidence of adequate response capacity. Indeed, in this case animals respond at higher than normal rates before ceasing to respond following pretreatment with dopamine antagonists (Yokel & Wise 1975, 1976; de Wit & Wise 1977; Ettenberg et al. 1982).

In the case of brain stimulation reward, responding is generally lower when animals are treated with dopamine antagonists; however, several conditions reveal that the low response rates are due to ineffectiveness of the reinforcer and not incapacitation of the animal. First, responding decreases progressively, both within sessions and across sessions, in animals pretreated with dopamine antagonists (Fouriezos & Wise 1976; Fouriezos et al. 1978; Franklin 1978; Franklin & McCoy 1979). When animals pretreated with moderate doses of dopamine antagonists are required to traverse an alleyway for access to the response lever that delivers the stimulation, performance is initially normal and deteriorates after several trials. Moreover, lever-pressing in the goal box deteriorates before running speed or latency to leave the start box; thus brain stimulation loses its ability to maintain responding in the goal box before the animals stop running to obtain it (Fouriezos et al. 1978). Second, after animals have stopped responding for brain stimulation reward under conditions of dopamine blockade, a reward-predicting environmental stimulus can, temporarily, reinstate normal performance; thus it appears that the decrement in responding results from the progressive loss of the expectancy of reward rather than from the immediate loss of response capacity (Fouriezos & Wise 1976; Franklin & McCoy 1979; Gallistel et al. 1982). Finally, when animals are tested in a ‘rate–frequency’ paradigm (an analogue of a dose–response paradigm), it is the stimulation frequency required to motivate the animal, rather than the maximum response rate that can be obtained from the animal, that is altered by dopamine blockade; animals can respond normally, but they require a higher stimulation ‘payoff’ if they are to keep doing so (Franklin 1978; Gallistel & Karras 1984). Finally, if dopamine-blocked rats are trained to earn rewarding brain stimulation in two ways (traverse a runway or press a lever) and are then tested in the two tasks sequentially, the animals initiate responding normally in the second task despite having ceased to respond normally in the first task; thus response capacity in the second task is unimpaired despite cessation of responding in the first task (Gallistel et al. 1982). Thus, while performance may be partially impaired by treatment with dopamine antagonists, data from several paradigms confirm that these drugs attenuate the ability of the stimulation to sustain normal performance before they interfere with the animal's capacity to generate such performance.

Importance of dopamine for food reinforcement

The concept of reinforcement is, at its core, a concept of how stimulus (Pavlov 1928) and response (Thorndike 1933) associations are formed and how they serve as the basis of habit acquisition (Skinner 1938). Food does not serve as a normal reinforcer in animals pretreated with dopamine antagonists; such treatment causes, for example, a dose-dependent decrease in how quickly animals learn to lever-press for food (Wise & Schwartz 1981). Under pretreatment with low doses of the dopamine antagonist animals eventually reach the normal performance asymptote; however, they require more trials to do so. With higher pretreatment doses learning is slower and may not reach the same performance asymptote. With yet higher doses there is no evidence of learning.

While the concept of reinforcement is most frequently used to explain response learning (Thorndike 1933; Skinner 1935; Hull 1937), it was first used in relation to stimulus learning (Pavlov 1928). Stimulus learning is now known to contribute significantly to response learning (Rescorla & Solomon 1967; Bindra 1972) and dopamine is thought to play a role in both (Wise 1989). Most studies of the reinforcing efficacy of food reward deal with the ability of the reward to maintain rather than to establish instrumental behaviour; without reinforcement both stimulus associations (Pavlov 1928) and response associations (Skinner 1933) extinguish.

When well-trained animals are tested under the influence of dopamine antagonists, food loses the ability to maintain normal responding. Whereas normal responding is initiated, responding slows progressively both within sessions and across sessions (Wise et al. 1978b; Dickinson et al. 2000). Similar progressive loss, both within and across trials, can be seen in the ability of food to maintain free feeding (Wise & Raptis 1986). The response slowing resembles what is seen in extinction conditions (when the normal reward is withheld), and is generally interpreted as a reflection of the impoverishment or ‘devaluation’ of food reward in the dopamine-impaired animal (Wise et al. 1978a,b; Xenakis & Sclafani 1981, 1982; Geary & Smith 1985). (See Salamone et al. (2005) for a dissenting opinion and Wise (2004a) for rebuttal.)

Few alternative hypotheses have been offered to explain the progressive response deficits seen when animals are tested under conditions of dopamine blockade. There is the suggestion that the progressive deficit might reflect a susceptibility to fatigue (or some other progressive within-trial performance impairment) caused by dopamine antagonists. This hypothesis can be ruled out from a variety of findings. First, the deficits are not only progressive within-trials; responding decreases progressively across repeated tests that are spaced days apart, with normal levels of responding between the days when the dopamine antagonist is given (Fouriezos et al. 1978; Wise et al. 1978b; Wise & Raptis 1986). Second, animals trained under intermittent dopamine blockade, like animals under intermittent reinforcement, respond more, not less, when tested for habit strength during extinction trials (Ettenberg & Camp 1986). There is no suggestion of any fatigue-like effect in this paradigm. Finally, when animals are trained in single daily trials to traverse a runway for food, dopamine blockade does not interfere with latency or running speed on the trial when the dopamine antagonist is given; rather, performance is impaired only on the following day, when the animals are free of the antagonist (McFarland & Ettenberg 1998). Fatigue due to dopamine blockade can explain neither the normal performance on the treatment day nor the slow performance on the day after treatment; here performance is impaired by the animal's memory of the previous day's experience and not by the pharmacological treatment itself.

Another suggested alternative for the progressive within-trial slowing of feeding and responding for food was that the slowing reflected the effects of enhanced satiety rather than the effects of blunted reward. This suggestion has been falsified in three ways. First, the same within-session progressive deficits are seen when dopamine-blocked animals are offered non-nutritive saccharin as when they are offered nutritive food reward; no such deficits are seen in control animals that do not receive the dopamine antagonist (Wise et al. 1978a). Second, within-session progressive deficits are seen when ingested sucrose is not absorbed but is, rather, drained through an open gastric fistula (Geary & Smith 1985). Third, the satiety hypothesis (like the fatigue hypothesis) cannot explain the fact that performance decreases across successive tests as a function of how much experience the animal has previously had with food in the dopamine-blocked condition (Wise et al. 1978b; Wise & Raptis 1986) and not as a function of experience with dopamine blockade in the absence of food (Wise et al. 1978b). Thus, it appears to be the memory of the food experience in the dopamine-blocked condition, not the experience of dopamine blockade itself, that determines the decline in responding between trials in the dopamine-blocked animal.

When animals are tested with a number of sucrose concentrations, dopamine-blocked animals respond to a given concentration as if it were weaker than normal (Xenakis & Sclafani 1981; Geary & Smith 1985; Bailey et al. 1986; Schneider et al. 1986, 1990). Thus, normal sucrose lick rates are shown in dopamine-blocked animals if the concentration of the sucrose solution is increased to 10% from the normal 5%. The devaluation of sucrose reward—the treatment of high concentrations as if they were lower—is seen either in animals pretreated with either D1- or D2-type dopamine receptor blockade (Schneider et al. 1986). Thus, like many dopamine-mediated behaviours (Clark & White 1987), performance for sweet reward appears to require co-activation of D1- and D2-type receptors.

The hypothesis that dopamine transmission is important for food reward implies that food reward elevates dopamine levels, as do, for example, some drug rewards (Hurd et al. 1989; Pettit & Justice 1989; Wise et al. 1995a,b; Ranaldi et al. 1999). Indeed, food reward (Hernandez & Hoebel 1988) and food reward-associated stimuli (Bassareo & Di Chiara 1999) do elevate dopamine levels in the nucleus accumbens. Indeed, just as μ and δ opiate agonists are rewarding in proportion to their ability to elevate dopamine levels (Devine et al. 1993; Devine & Wise 1994), so are different sucrose concentrations rewarding in proportion to their ability to elevate dopamine levels in the nucleus accumbens (Hajnal et al. 2004).

Recent issues

An important role for dopamine in reward function has been well established for many years, but several fine points continue to be discussed in the literature. Is dopamine absolutely necessary for reward? Is dopamine more important for the expectancy of reward before it is delivered or for the impact of reward after it is delivered? Is the dopamine in nucleus accumbens more important for reward than the dopamine in other brain regions? Some of these recent issues are best resolved by consideration of the early literature.

First, studies involving pharmacological blockade of dopamine receptors have suggested a necessary role for dopamine in the reward function (Wise & Rompré 1989; Wise 2004a). Recent studies with genetically engineered mice challenge this strong position. First, deletion of the tyrosine hydroxylase (TH) gene with rescue of noradrenergic function results in mice that are born superficially normal and eat and gain weight for 10–15 days, at which time their eyes normally open and they begin nibbling and foraging for solid food. Unless treated with L-DOPA, they then lose weight, usually dying by 4 weeks of age (Zhou & Palmiter 1995). If treated with L-DOPA, however, they are alert and active for about 8h after their daily treatment, eating enough during this period to maintain themselves (Szczypka et al. 1999). Restoration of TH expression in the caudate nucleus but not in the nucleus accumbens is sufficient to restore normal feeding in the TH knockout animals (Szczypka et al. 2001).

If these dopamine-deficient mice are maintained by daily L-DOPA treatment but tested when showing Parkinsonian akinesia, 18 or 28h after the previous L-DOPA maintenance injection, they show normal sucrose and saccharin preferences over water, and drink more of these solutions than water in single bottle tests even after any trace levels of residual dopamine are purged by treatment with the dopamine-releaser amphetamine (Cannon & Palmiter 2003). Such animals show between-session (but not within-session) learning in a water escape task (Denenberg et al. 2004). If aroused with caffeine, such animals can learn a side-preference (but not reverse it) for food reward in a T-maze (Robinson et al. 2005). These findings establish that, whatever its importance in normal animals, normal dopamine function is not an absolutely necessary condition for rudimentary instrumental learning.

Another recent issue is whether dopamine is important for the motivation to seek anticipated food or rather for the reinforcing effects of food once it has been earned and received (Berridge & Robinson 1998; Salamone & Correa 2002). Food rewards have both kinds of effect (Wise 1989, 2004b). The primary effect would appear to be the ability to reinforce learning, as evidenced by: (i) the fact that animals do not learn food-seeking responses when their dopamine systems are blocked (Wise & Schwartz 1981); (ii) the fact that the effects of finding a piece of food when the dopamine system is blocked are more evident on the day after the dopamine blockade than on the day of the dopamine blockade (McFarland & Ettenberg 1998); and (iii) the fact that food-seeking habits extinguish when animals are tested under dopamine blockade (Wise et al. 1978a,b).

However, secondary to its role in the reinforcement history of the animal, dopamine clearly does have a role in the motivation of reward-seeking behaviours. While food-seeking is thought to be initiated by hunger, it is food-predictive or ‘incentive–motivational’ environmental cues in the environment that release and guide the behaviour. The incentive–motivational salience of these stimuli depends upon the prior dopamine-dependent reinforcement of their association with the reward. Again, the McFarland & Ettenberg (1998) study clearly illustrates the point. Their trained animals left the start box promptly and ran the runway quickly except on the day after they obtained food in the goal box while under the influence of the dopamine antagonist haloperidol. Thus incentive–motivation—the process by which reward-predictive cues activate and motivate an animal—depends on a dopamine-dependent history of association (reinforcement) between the cues and the reward they predict.

In addition, a ‘priming’ presentation of a reward sample (rather than of a conditioned predictor) can arouse an animal and motivate it to seek more of the priming stimulus. Salted peanuts and potato chips are good examples of rewards that are particularly good at priming further reward-seeking. A taste of such rewards is particularly effective at renewing reward-seeking behaviours after they have been given-up because they are no longer rewarded (Skinner 1938). Administering dopamine agonists is among the most effective ways to reinstate extinguished reward-seeking (de Wit & Stewart 1981, 1983; Wise et al. 1990).

Questions about the role of dopamine in reward function have also arisen from evidence that dopamine seems unimportant for the facial responses to oral presentation of sweet tastants (Berridge et al. 1989; Pecina et al. 1997). If one was to assume correspondence between the facial expression of ‘liking’ food and the ability of that food to serve as a reinforcer, this finding would pose a challenge to the view that dopamine is important for the ability of food to stamp in stimulus and response associations. The assumption of correspondence between the facial expression of liking and the hedonic response to reinforcement is, however, open to serious question.

Initial studies of the orofacial responses of humans to various tastants identified unique reactions to sweet, sour, and bitter stimuli (Steiner 1973). In rodents, the reactions to sweet and bitter stimuli are clearly distinguishable and have been classified as ‘ingestive’ and ‘aversive’ fixed action patterns (a misnomer, as it is the stimulus, not the action pattern, that is aversive) (Berridge & Grill 1984). The rodent orofacial responses to sweet and bitter tastants are, essentially, the licking of lips associated with the acceptance of a fluid and the gagging and chin-rubbing associated with fluid rejection, respectively. Following Schneirla's (1959) argument that approach and withdrawal responses are the only objective terms applicable to all motivated behaviour in all animals, Berridge & Robinson (2003) have suggested that these oral fixed action patterns reflect the hedonic assessments—liking or disliking—of various tastants. What is not clear, however, is how well the fixed action patterns of ingestion and rejection, expressed in decorticate rats (Grill & Norgren 1978; Hall & Bryan 1981) and anencephalic children (Steiner 1973), correlate with the higher-level subjective hedonic responses to and objective reinforcement induced by various foods.

The recent arguments of Berridge & Robinson (1998) that dopamine is important for the wanting of rewards but not the liking of rewards is based on several assumptions about the relation of the taste reactivity test to generalized emotional states. First, there is the assumption that the brainstem reactions to taste stimuli determine the hedonic response to those stimuli (Berridge 2000). This assumption begs the question of how humans learn to like bitter tastants like coffee and broccoli. Second, there is the assertion that the liking of a tastant need not be conscious (Berridge & Robinson 1998; Berridge & Winkielman 2003); in this view, whether the subject likes or dislikes a tastant is more directly evident to an outside observer than to the subject itself. This may well be true, but the possibility raises the question of why subjective lay terms like ‘wanting’ and ‘liking’ should be substituted for the more traditional motivational labels ‘drive,’ ‘incentive–motivation’ and ‘reinforcement.’ Third, there is the implicit assumption that all rewards are pleasant; this assumption is falsified by the fact that animals can be trained to work for aversive footshock (Kelleher & Morse 1968) and that initial injections of heroin, while extremely habit-forming, are often reported to be aversive (Haertzen 1966). Finally, there is the assumption that the mechanism of wanting and liking of tastants can be generalized to other rewards such as sexual interactions and addictive drugs (Robinson & Berridge 2003).

In order to see the relevance of subjective wanting and liking for the behavioural control by the motivating and reinforcing effects of food reward, it is important to distinguish between the wanting and liking of an abstract concept such as ‘sweets’ and the wanting and liking of a specific food morsel that is currently available to the peripheral senses. While Berridge & Robinson (2003) hold that you can both want and like a given tastant (like chocolate fudge) at the same time, you cannot experience at the same time the wanting and liking of a given specific morsel of food. If you do not yet have the morsel you can want it without knowing for certain that you will like it; if you have it in your mouth you can like it but it is no longer available to want. Inasmuch as it is a real morsel and not an imagined category that controls behaviour at a given time, we can, for behavioural analysis, identify wanting with the state of mind of an animal prior to earning a given food morsel and identify liking as the state of mind once the reward has been earned and is being sensed. From this perspective, the animals in the McFarland & Ettenberg (1998) study discussed above want the food pellet before they have tasted it on the haloperidol treatment day. If there is a deficit in the wanting of food in these animals, it is a deficit in wanting the next pellet, the one offered on the day after the pellet eaten during haloperidol treatment. In as much as the animals ran normally on the haloperidol treatment day and failed to do so on the day following the haloperidol treatment, it seems an inescapable conclusion that they wanted the food despite dopaminergic dysfunction. This separation of food-seeking into discrete trials allows us to see that haloperidol treatment disrupts food-seeking only after the animal has had the opportunity to taste the food under the influence of the dopamine blocker. As discussed above, it would appear that the importance of dopamine for the wanting of food on a given day exposure results from the role dopamine played in the prior liking of food on earlier exposures (whether it be days, trials, morsels, or bites). It is the prior liking of (or reinforcement by) a foodstuff—a dopamine-dependent function—that establishes a subsequent craving for that foodstuff (Rozin & Kalat 1971).

Another challenge of the view that dopamine plays an important role in food reinforcement stems from the unwarranted assumption that all the reward-relevant dopamine functions occur in nucleus accumbens (Salamone et al. 2001). While a good deal of work implicates nucleus accumbens in the rewarding effects of psychomotor stimulants (Roberts et al. 1977; Ikemoto et al. 1997; Roberts et al. 1980), and while it is nucleus accumbens dopamine fluctuations, for the most part, that have been correlated with drug reward and food reward (Di Chiara & Imperato 1988; Hernandez & Hoebel 1988; Hurd et al. 1989; Pettit & Justice 1989; Wise et al. 1995a,b; Ranaldi et al. 1999; Bassareo & Di Chiara 1999), and while protein synthesis in nucleus accumbens impairs instrumental learning for food reward (Baldwin et al. 2002), it nonetheless remains the case that lesions of the dopamine projection to nucleus accumbens do not cause feeding deficits (Ungerstedt 1971; Ervin et al. 1977), while lesions of the dopamine projections to the dorsal striatum do (Ungerstedt 1971). Moreover, it is genetic restoration of dopamine function in the dorsal striatum, not in nucleus accumbens, that rescues feeding in dopamine-deficient knockout mice (Szczypka et al. 2001). While lesions of nucleus accumbens disrupt cocaine self-administration more than they disrupt heroin self-administration (Pettit et al. 1984), ventral tegmental lesions that damage nigrostriatal as well as mesolimbic fibres disrupt both behaviours (Bozarth & Wise 1986). While nucleus accumbens injections of opiates (Olds 1982; Goeders et al. 1984) or psychomotor stimulants (Hoebel et al. 1983; Carlezon et al. 1995; Carlezon & Wise 1996; Ikemoto et al. 1997) are rewarding, injections of cocaine into the medial prefrontal cortex (Goeders & Smith 1983, 1986) or olfactory tubercle (Ikemoto 2003), or injections of opiates into the ventral tegmental area (Bozarth & Wise 1981; Devine & Wise 1994; Zangen et al. 2002), are also rewarding. Thus, nucleus accumbens is not the exclusive seat of dopamine-dependent reward function, and nucleus accumbens lesions should not be expected to disrupt all rewards.

Another recent issue is whether activation of dopamine neurons represents reward, prediction of reward, or an error signal reflecting the difference between earned and expected reward. The issue arises from electrophysiological studies of Schultz and collaborators, suggesting that dopaminergic neurons respond to rewards as long as they are not fully predictable, but transfer to conditioned stimuli predicting reward once the predictive significance has been learned (Schultz 1986; Romo & Schultz 1990; Ljungberg et al. 1991, 1992; Schultz et al. 1993). It is naive, of course, to expect that dopamine would play a specialized role in only one of these functions.

Moreover, Schultz's (Schultz 2002) distinction between ‘reward-predicting’ and ‘rewarding’ stimuli in these studies merits closer analysis. First, the rewarding event is not consistently defined in the various studies of Schultz's monkeys. In these studies, the reward was sometimes identified with presentation of the rewarding object rather than with the oral contact with the reward or with the post-ingestional consequences of that reward that truly constitute the rewarding event. In some studies, the primary rewarding stimulus was a piece of apple presented in a cup at arm's reach; in others it was a drop of fruit juice presented in a spout near the animal's mouth. In the case of the juice, the rewarding event was assumed to be the delivery of the juice rather than the taste of the juice; presumably, in this case, latencies were short enough that the presentation of the reward and the tasting of the reward were almost concurrent. In the case of pieces of apple, however, dopamine responses were noted when the monkey touched the food and again when the monkey tasted the food. Here, the tactile rather than the taste contact was taken as the rewarding event. Whether receipt of reward was defined by the touch of the apple pieces or the delivery of the juice, dopamine neurons responded to what was designated as the rewarding event in early stages of training but to what was designated as the reward-predicting event (sound and sight of latch opening in the case of the apple reward and sight of illumination of a light cue in the case of the juice reward) and not, after a great deal of training (thousands of training trials) to the ‘reward’ itself.

Clarification of the influence of training in these studies comes from subsequent studies in which the effectiveness of the reward-predictive cues was varied. When visual or auditory cues, or both, predicted reward with 100% certainty, the responses of dopamine neurons shifted from the reward itself (as defined above) to the reward-predictive environmental stimuli (Schultz 1986; Romo & Schultz 1990; Ljungberg et al. 1991, 1992; Schultz et al. 1993; Fiorillo et al. 2003). In the case where presentation of the reward was predicted by both auditory and visual stimuli, the responses of dopamine neurons to the reward-predictor (latch-opening) was weakened when the visual stimulus (sight of the door) was occluded (Schultz 1986). When a visual stimulus predicted reward with 75, 50, or 25% probability, responsiveness to the visual stimulus decreased and responsiveness to the tactile or taste stimulus increased accordingly (Fiorillo et al. 2003). This finding suggests the need for a closer examination of the distinction between reward and reward-prediction (Wise 2004b). While food might be considered a ‘primary’ reward (Schultz 1986), food is identified by each of the five senses and it is only in the taste of sweet foods (and perhaps the taste of salt in the case of sodium deficiency; Quartermain et al. 1967) that a strong argument can be made that the sensory experience of food is innately rewarding (Steiner 1974; Hall & Bryan 1981) in the absence of learned association with its post-ingestive consequences (Le Magnen 1959; Rozin & Kalat 1971; Messier & White 1984; Sclafani 2004). To the degree that it is the post-ingestional effects of a food that is reinforcing, however, as is the case with the rewarding effects of minerals or vitamins for deficient animals, the sensory experience of a given foodstuff becomes a reinforcer in its own right: a conditioned reinforcer (Robbins 1978). Certainly, the touch of a piece of apple is a learned reinforcer for Schultz's very well trained monkeys (Schultz 1986), as, it would appear, are the click or sight of the opening door behind which food is to be found (Schultz 1986). Even the sweet taste of saccharin appears to be a learned reward (Messier & White 1984). Thus, Schultz's distinction between reward-predictors and primary reward is fuzzy distinction; the visual or auditory awareness of the availability of food, if it is a 100% predictor, is certainly as primary as the tactile awareness of that availability, and it is arguably as primary as the olfactory or gustatory awareness of foods that satisfy a mineral deficiency. The fact that dopamine neurons no longer fire in response to the taste of apple when that taste has been predicted by the feel of the apple or the click that predicts the feel is completely consistent with the fact that food-predicting stimuli can become conditioned reinforcers in their own right so long as dopamine function is not impaired during the association of the conditioned stimulus with the food (Beninger & Phillips 1980; Taylor & Robbins 1986).

This does not negate the fact that the phasic activation of dopamine neurons occurs in proportion to the discrepancy between the expected reward and the observed reward, or that such information participates in the learning associated with reinforcement. It should be noted, however, that the short-latency activation of dopamine neurons by visual stimuli occurs before the eye moves to fixate a peripheral visual stimulus. Thus dopamine neurons are likely to be only reporters of the discrimination made by the inferior colliculus between reward-predictive or otherwise salient stimuli and various stimulus events that bear no relation to rewarding events (Dommett et al. 2005).

Conclusions and perspectives

Brain dopamine plays several roles in the ability of food to serve as a reward. It is important—but apparently not completely necessary—for the reinforcement function of rewards, their ability to stamp in stimulus and response associations. Current evidence suggests that dopamine in the caudate nucleus may play a more important role than dopamine in the nucleus accumbens in the reinforcement of response habits (White & McDonald 2002; Wise 2004a). Past dopamine-dependent reinforcement of stimulus–reward associations is, in turn, important for the incentive–motivational energizing effect of reward-predictive cues in the environment. And, of course, some degree of basal dopamine is important for rudimentary behaviour of any kind (Hornykiewicz 1979; Stricker & Zigmond 1985). This does not mean that dopamine plays a specialized or exclusive role in reward function. Other neurotransmitter systems are certainly involved and it is not clear what subsets of dopamine neurons contribute to reward function. Reward function—and food reward in particular—is only one of the many functions in which dopamine plays an important contributing role.

Schedules of Reinforcement

In the Skinner-box it is possible to change the contingency between the responses and the delivery of reinforcement so that more than one response may be required in order to obtain the reward. A whole range of rules can govern the contingency between responses and reinforcement - these different types of rules are referred to as schedules of reinforcement. Most of these schedules of reinforcement can be divided into schedules in which the contingency depends on the number of responses and those where the contingency depends on their timing.

Schedules that depend on the number of responses made are called ratio schedules. The ratio of the schedule is the number of responses required per reinforcement. The "classic" schedule, where one reinforcer is delivered for each response, is called a continuous reinforcement schedule - it has a ratio of 1. A schedule where two responses have to be made for each reinforcer has a ratio of 2 and so on. A distinction is also made between schedules where exactly the same number of responses have to be made for each reinforcer - fixed-ratio schedules, and those where the number of response required can differ for each reinforcer around some average value - a variable-ratio schedule. A schedule where exactly 20 responses are required for each reinforcer is called a fixed-ratio 20 or FR20 schedule. One where on average 30 response are required is called a variable-ratio 30 or VR30 schedule.

If the contingency between responses and reinforcement depends on time, the schedule is called an interval schedule. Reinforcing the first response an animal makes after the discriminative stimulus (SD) light has been on for 20 seconds and ignoring responses it makes during that 20 seconds would correspond to such a schedule. Where the interval which must elapse between the onset of the SD and the first reinforced response is the same for all reinforcers the schedule is called a fixed-interval or FI schedule. Again, the intervals could also vary around some average - this is called a variable-interval or VI schedule. It s possible to combine these schedules in various ways and even to construct other basic types of schedule (e.g. ones where animals are reinforced for maintaining specified intervals between responses - differential reinforcement of low rate of response or DRL schedules). The important thing about these different schedule, however, is the differences in response patterns and learning that they produce. These differences may tell us about part of what is learned in operant conditioning. A summary of the basic types of schedules might be useful:

• Fixed-Ratio (FR) in which the first response made after a given number of responses have been in the presence of the discriminative stimulus is reinforced. For example on an FR 15 schedule every 15th response is reinforced.
• Fixed-Interval (FI) in which the first response made after a given time interval is reinforced. For example, on an FI 20 sec. schedule the first response made after 20 seconds from the onset of the discriminative stimulus is reinforced. The discriminative stimulus would normally then be turned off during the period the animal consumes its reinforcer.
• Variable-Ratio (VR) is similar to FR except that the number of responses required varies between reinforcements. On a VR 15 schedule 15 responses are required per reinforcer on average, but one reinforcer may only require 3 responses while the next is obtained after 22 responses.
• Variable-Interval (VI) is similar to FI except the interval requirements vary between reinforcers around some specified average value.

The most characteristic response patterns are produced by FI and FR schedules. Responses in operant-conditioning experiments were traditionally recorded using a pen recorder (see figure in the Skinner-Box document) in which a pen was drawn across paper at a constant rate, the pen was moved up a small amount each time an animal made a response, a larger diagonal movement recorded the occurrence of reinforcements. Animals produce constant rate responding to FR schedules with a distinct pause in responding after each reinforcement.

The rate of response is inversely proportional to the ratio requirement. The length of the post-reinforcement pause in responding also increases as the ratio increases. The pattern of animal responding on FI schedules is quite different. After each reinforcement animals respond on FI schedules with gradually accelerating response rates which produces a 'scalloped' record:

The main feature of variable schedules is that, in animals, ratio schedules produce larger response rates than interval schedules for the same reinforcement density. For example, one animal might be trained on a variable ratio schedule and the times at which it received reinforcement could be noted. These time could then be used to form a 'yoked' variable interval schedule for another animal - an interval schedule where the interval between SD onset and the onset of a response-reinforcement contingency is determined by the times at which the first animal received each reinforcement. Typically the second animal would produce much slower response rates on the yoked schedule even though the frequency of reinforcement received by the two animals was more or less the same.

Intro To How Dog Training Works

By Hannah Harris

Although dogs have helped people with specific jobs for millennia, today most resemble family members more than employees. According to the American Pet Products Manufacturers Association, in 2005, American pet owners spent an estimated 39.5 billion dollars on their pets, more than twice what they spent in 1994.

While this represents a golden age for lucky dogs, the status of family member may also require that the dog meet certain standards of behavior. Although you might guess otherwise from the abundance of doggy fashions now available, dogs are not tiny, furry people. They have their own way of thinking and doing things. Thousands of dogs are surrendered to animal shelters each year, or permanently relegated to a backyard pen, simply for acting like dogs.

Dogs and people can live together happily, but this requires that owners make the effort to bridge the species gap and train their dogs to behave appropriately in human society. There are many different ways to train dogs and just as many trainers who will say that their way is the only "right" way, but the reality is that there are multiple methods that all work. The main difference between them is how quickly they work, and how enjoyable they are for dog and handler.

In this article, we'll explore the history and ideas behind most methods of dog training and talk about one of the most popular methods today: Clicker training.

Learning Theory

Dog training typically centers on operant conditioning. The first scientist to define this concept was B.F. Skinner, who studied the work of Russian physiologist Dr. Ivan Pavlov on animal behavior. In Pavlov's groundbreaking study, dogs learned that a stimulus (in this case, a bell) meant they were about to be fed. Starting with two things that are naturally paired -- salivating and being fed -- Pavlov added a third component by ringing a bell before feeding. After a few trials, the dogs learned to associate the bell with being fed and would react by salivating at the sound of the bell in anticipation of their food but without any food present.

Pavlov's Famous Study

Since dogs naturally begin salivating when offered food, food is an unconditioned stimulus. No conditioning or special training is necessary to cause the dog to salivate, which is an unconditioned response. In contrast, a ringing bell does not normally cause dogs to salivate; they will do so only if they have been conditioned to associate a bell with being fed. Therefore, the bell is a conditioned stimulus. The dog's new reaction is a reflex to the stimulus and is a conditioned response.

Many of us see this today with our own dogs when they break into a frenzy of barking at the sound of the doorbell, sometimes even a doorbell on television. In this case, the dog has been conditioned to associate the stimulus of the bell with the imminent arrival of a stranger.

When we see flashing lights or hear a siren behind us while driving, we may reflexively tense up and our heart rate may increase. We have been conditioned to associate the sound of sirens with the unpleasant and stressful experience of getting a ticket. This is classical conditioning. Both animals and people can learn to relate a pair of events and respond to the first in anticipation of the second. This type of learning is passive and involuntary; it occurs without the learner doing anything and often without awareness.

While Pavlov's work dealt with a reflexive reaction to a conditioned stimulus, Skinner became interested in creating a specific behavioral reaction to a stimulus by adding a reinforcer. A reinforcer can be either a reward or a punisher. A reward is anything that increases the frequency of an action; a punisher is anything that decreases its frequency.

When we are rewarded for a certain behavior, we are likely to repeat that behavior. When we are punished for a certain behavior we are likely to stop. This type of learning is active and voluntary; it depends on the actions of the learner.

Because the definition of a reinforcer is based on its effectiveness, it's important to remember that a reward for one person may not be meaningful, and thus not a reward, for another. Similarly, what is a reward in one context may not be somewhere else.

A rat in a basic Skinner box

Skinner showed that both animals and people would perform certain behaviors for a reward. In his experiments with rats and pigeons, Skinner showed how animals could learn to press a lever to get a food reward. When the animals were first introduced to the test box they moved around randomly. When they accidentally depressed the lever, a food pellet was dispensed. They quickly learned to depress the lever on purpose to get a pellet. He also shaped behaviors that are more complicated by reinforcing them step by step. Skinner called his approach "operant conditioning" because the animal's behavior actually operated on the environment (pressing the lever) in response to the anticipated outcome (getting a food reward).

Reinforcers

Reinforcers can involve either the addition of a new element or the removal of an element currently present. The terminology for this is a little confusing, but adding something is referred to as "positive," though not necessarily in the sense of "happy" or "good." "Negative," in this case, is the removal of something, and doesn't necessarily mean "bad." Therefore, both rewards and punishers can be either positive or negative.

Giving a parrot a piece of fruit for waving its foot is an addition of something good (a positive reward); a horse moving faster to stop the pressure of spurs is the ending of something bad (negative reward). Even though "negative reward" sounds like an oxymoron, the removal of something bad is a kind of reward.

Punishers work exactly the same way. When a dog pulls on the leash and gets a sharp tug in the opposite direction, especially when using a choke chain or prong collar, it's a positive punisher or correction; the dog receives unpleasant feedback for undesirable behavior. Alternatively, a punisher can be the removal of something good -- as when a child loses the privilege of going out with friends following misbehavior; this is a negative punisher.

Reinforcers can be almost anything as long as they are meaningful to the dog. One dog may think treats are more valuable than toys, while another may feel the opposite. It doesn't really matter what the reinforcer is, but for practical reasons, some reinforcers are easier to work with than others. Also, the same reinforcer doesn't have to be used every time or in every situation. Some tasks may require a more valuable reinforcer than others. Your dog may work for one type of reward in the relative calm of your home but may need something more desirable to maintain focus in class.

Markers

Keller and Marian Breland were students of B.F. Skinner and expanded his techniques to train a variety of different kinds of animals. In the 1950s, Keller Breland began developing a training program for marine mammals. For obvious reasons, it is difficult and dangerous to design an effective punisher for a dolphin or an orca. It is also challenging to reward a marine mammal promptly because they are in the water, and the trainer is often some distance away on land.

Photo courtesy Paul Anderson / MorgueFile

Many of the same problems are inherent in training dogs. If a dog sits, then jumps up, spins around and is given a treat, it will probably not know which part of the performance pleased the handler. This is especially true if it took a minute for the trainer to pull out the treat and present it to the dog. Typically, the dog will pair the reward with the final behavior that it performed before it got the treat. So if the dog sat, then jumped up and got a treat, what it's really being trained to do is jump up, not sit.

This is also true of punishers. If a dog runs away from its owner and engages in a game of hide-and-seek, the owner might punish the dog when he catches it. However, the last thing the dog did before being punished was come to the owner. So coming when called is the behavior that is likely to decrease, instead of running away.

Breland solved this problem by designing a marker, or signal, that would let the animal know it had performed correctly and would be getting a reward shortly. Breland used classical conditioning to pair a marker signal with a reward, so that when the animal heard the signal it knew it would be getting a reward. Then he used operant conditioning to shape behavior using positive rewards.

The marker helps to reinforce the correct behavior because it is immediate. The marker is not the reward; it is simply a signal that the behavior was correct and a promise of a reward to come. Because marine mammals are naturally oriented towards communication via sound, it made sense to use a whistle blast as the marker.

Karen Pryor used the same positive reinforcement techniques in the 1960s to train dolphins. She realized the broad applications of this type of behavioral modification and in 1984 she authored the book "Don't Shoot the Dog," which, despite the title, is not really about dog training. It covers using positive reinforcement to shape the behavior of anyone from pet cats to difficult teenagers. Many businesses still use this book to teach their employees about effective management.

Pryor used a metal clicker as a marker to begin shaping the behavior of dogs, as well as many other animals and hers is the name most commonly associated with modern clicker training. Her techniques were adopted by other trainers and with the advent of the Internet, clicker training spread rapidly.

Karen Pryor describes the click as taking a picture of the behavior that you want; you snap it in that second. The click means, "you did something, it was the right thing, and you will be getting a treat for it."

Many beginning trainers make the mistake of clicking to mark a behavior, but then not following the click with a treat. With no actual reward, the dog may continue to offer the behavior for a while but it will eventually disappear.

Other Markers

A marker doesn't have to be a clicker. It can be anything that marks the desired behavior. Dolphin trainers often use whistle peeps. Fish can be trained by flicking a flashlight on and off, and vibrating collars can mark behavior for deaf dogs. Some people mark behavior with a particular word, like "yes!" The marker can be anything that is short, specific, and consistent. On those grounds, care must be taken when using the voice as a primary marker because tone is often inconsistent and marker words or sounds may also be used in regular conversations and lose meaning for the dog.

Clicker Training: Introducing Cues

The clicker is not inherently meaningful to the dog. Like Pavlov's bell, the dog must learn that it means, "Treats are coming!" via classical conditioning. To do this, trainers "charge" the clicker by repeatedly clicking and then immediately offering a treat. In this way, the dog learns to pair the clicker with the treat. Once the dog knows that a click means a treat, it is ready to start learning new behaviors.

Trainers vary in their methods of eliciting a behavior. Some advocate using food to lure the dog into position. Others simply wait for the dog to offer the behavior simultaneously. Most clicker trainers do not advocate physically pushing the dog into position, as that is counter to the force-free philosophy of clicker training.

Once the dog offers the behavior, timing is critical. The trainer must click at the exact moment that he sees the behavior he wants. If the dog lies down and then rolls over before the handler clicks, rolling over has been marked as the desired behavior, not lying down.

Dogs can learn complicated behavior patterns using clicker training if you teach the sequence gradually. For example, if you wanted to train your dog to jump through a hoop, you might initially click and treat the dog just for walking up to the hoop. Once the dog is reliably walking up to the hoop, you would click only when it stuck its head through the opening, and then only when it walked through. Finally, you would click only when the dog actually jumped through the hoop. The standard for what will earn a reward keeps getting higher as the dog learns each new step. This is shaping.

Rather than giving a cue and then teaching the dog what it means, most clicker trainers prefer to introduce the cue only after the dog is reliably offering the behavior. Luring motions (such as holding a treat and moving it in front of the dog's nose, then to the ground to teach a dog "down") can be adapted into hand signals for cues by stylizing the motion and eliminating the food lure. Many trainers feel hand signals are easier for dogs to learn that verbal signals anyway, but having a dog that responds to either is ideal. Once a dog is offering the desired behavior, the handler can begin using the cue so that the dog learns to associate the two. Eventually, the handler will only click the behavior if it was requested with a cue, not when it's offered spontaneously.

It's important to remember that animals are contextual learners. That means that they may understand a cue in one place but not another. A dog may be able to sit flawlessly when the handler is standing, but become very confused when the handler gives the cue from a sitting position. When training a new cue handlers need to add new contexts, backing up when necessary, to help the dog generalize.

Clicker Training: Eliminating Undesirable Behavior

It is much easier to teach a dog to do something good than it is to teach it not to do something bad. When breaking a dog of an undesirable behavior, the first thing to consider is what reward the dog is getting for it. He must be getting some reward or he wouldn't keep doing it, but sometimes rewards are subtle or counterintuitive. When a dog jumps up on his owner and the owner pushes him off, being handled is the physical reward and may be more powerful for the dog than the punishment of yelling. Handlers must be careful not to accidentally reward dogs for undesirable behavior. Once the handler identifies and eliminates the accidental reward (to the extent possible), the next step is often to train an incompatible behavior. The dog trained to sit when the owner picks up the leash cannot simultaneously jump up and knock her owner over.

The clicker is a training tool and handlers should not use it indefinitely. The purpose of the clicker is to communicate the desired behavior. Once the dog understands the cue and performs it reliably, you can eliminate the clicker. You can still reward the dog, but over time you may shift from a highly prized food reward to a less desirable food reward, then perhaps to just a verbal reward. When the behavior is learned completely, no reward may be necessary at all, though dogs as well as people appreciate feedback for a job well done.

Operant conditioning is a powerful tool in shaping the behavior of almost any animal, including dogs. A focus on positive reinforcement helps everyone enjoy training and deepens the bond between the trainer and learner. The use of a marker helps accurately pinpoint the desired behavior and greatly speeds the training process. When a dog fails to learn something, it's often because of a breakdown in communication rather than an unwillingness to cooperate. A good trainer can help troubleshoot these problems. The only real limits of this kind of training are the ability of the handler to identify meaningful reinforcers correctly, and to break the desired behavior into manageable steps.

Rewards vs. Punishers

Why use rewards instead of punishers? There's no question that both rewards and punishers can be effective reinforcers, but rewards are generally easier to use. To be effective a punishment must be immediate, intense, unavoidable, and consistent, which is surprisingly difficult to achieve in real life. It is very easy to mess up a punisher by using poor timing, excessive or inadequate force. In addition, unpleasant consequences are often associated with the handler rather than with the misbehavior.

via how stuff works

Learned Helplessness

By Dr. Beverly Potter

Psychologist Martin Seligman spent years studying the impact of "controllability" on people and animals which is described n his book, Learned Helplessness: On Depression, Development and Death.

In a typical study matched pairs of dogs were divided into two groups, one where the dog could control what happened and one where it could do nothing. In the first situation, a naive dog was place in a room with an electric grid floor. This first situation was called "controllable" because the room also contained a puzzle. If the dog "solved" the puzzle, the shock stopped. In this example the puzzle was a lever, which when pushed, turned off the shock.

Since the dog had never been in the room before and it had no knowledge of the shock it was about to receive, the dog was relaxed and friendly as it wagged its tail and wiggled its nose. However, when the electric floor was activated, the dog's demeanor changed dramatically. It jumped and yelped as it frantically searched for a way out. In the process the dog accidentally pushed the lever, causing the shock to stop -  a powerful negative win. Over the next couple of trials when the dog was put back in the room and the shock turned on, the dog learned very quickly to run to the lever and push it. The dog was highly motivated - albeit avoidance motivation - because the dog learned that it could do something to control its world.

The dog in the "uncontrollable" group was placed in the same room with the electric floor, only this time there was no puzzle and there was nothing that the dog could do to turn off the shock. Just like the first dog, it ran around trying to find a way out. When the dog eventually learned that there was nothing that it could do it gave up, and laying down on the floor, it took the shock. The dog was not motivated because it learned that it was helpless.

Later the second dog that had learned that it was helpless was put into the room with the puzzle but it made no effort to find a way out. Instead the dog just lay on the floor and took the shock. Even when the door was left wide open, the dog did not attempt to escape the shock. The dog could not seem to learn that the conditions had changed and that it was no longer helpless.

To summarize, the second dog "learned" that it was helpless and stopped trying to get away. Its motivation to escape was extinguished or eliminated. In the process, dog exhibited a lot of negative emotions: first yelping and growling, later whimpering, and eventually just remaining motionless. Something happened that interfered with the dog's ability to learn when things changed and when it could do something. In effect, the dog burned out.

Powerlessness at work can affect people in the same way. As you learn that there is nothing you can do you'll probably experience negative emotions, beginning with frustration and anger, later anxiety and guilt, and eventually depression and despair. In the process, motivation declines. When the conditions change you will probably find it hard to learn and continue acting helpless.

Of course, scientists can't subject people to such experiments so we have no direct scientific data on the effects of powerlessness on human subjects. However, we can speculate that the battered-wife syndrome may be caused by learned helplessness, for example. If the woman believes that she is powerless before an abusive husband, she will probably act like the dog on the grid floor, taking the abuse and not running away when she has the opportunity. People in the ghetto who don't avail themselves of opportunities, such as educational programs, may fail to do so not because of laziness, but because they have learned that they are helpless and, as a result cannot act. Homeless people who are skilled and were once securely employed but now are
unable to hold a job, may also be victims of learned helplessness. People who are chronically depressed may have become so as a result of uncontrollable situations. For example, there is a statistic claiming that more Vietnam Vets committed suicide than were killed in the war. Perhaps they were suffering from burnout. Remember the yellow ribbons and the people held hostage over 400 hundred days in Iran? A large percentage of the hostages developed chronic depression. Perhaps it was learned helplessness.

In his research, Seligman discovered that animals who learn to be helpless have little resistance to adverse situations. They often die in as few as ten minutes when placed in a survival situation, whereas animals who have learned mastery continue fighting to survive hours later. This research suggests learned helplessness is literally life-threatening. Research suggests it even triggers a biological suicide mechanism. In some cases biological functions simply slow down or cease; other studies indicate that the body may develop a terminal disease. This notion has been supported by research with cancer patients that suggested that people who are depressed and feel like victims were more likely to get cancer.

An uncontrollable situation can be harmful without being physically painful, however. Feeling helpless can do serious damage to motivation in any situation, even those filled with luxury and privilege. An example is the poor little rich boy whose daddy does everything for him.  As a kid he breaks a window with a ball, and daddy fixes it.  He gets ho hum grades in school but gets into college anyway because daddy gave a big donation. After graduation he gets a job with a big salary and a corner office in daddy's firm.

Surprisingly, the poor little rich boy's situation is similar to the unhappy worker suffering under a hypercritical boss.  While the worker is overloaded with criticism and the rich boy has an overabundance of goodies, both lack a sense of control. Neither feel they can influence what happens to them. Seligman emphasizes in his research on learned helplessness that it is not the quality of the situation that causes feelings of helplessness and depression. Even though we tend to think that the cause is punitive circumstances, situations filled with rewards can also lead to the same debilitating learned helplessness and depression when the person does not have to perform to get those rewards. Seligman describes research with rats and pigeons in which they could choose between getting food free and having to make certain responses to get the same food. The rats and pigeons choose to work!

via Docpotter

Separation Anxiety

The term “Separation Anxiety” is used to describe or explain destructive behaviour associated with the fear of being left alone.  There are mild forms of separation anxiety, which can generally be treated with behaviour modification exercises; and there are severe cases, often requiring medication.  Here I will discuss the symptoms and some common behaviour modification techniques.

Reading the signs

There are several different behaviours that could signal separation anxiety.  Some dogs will eliminate in the house when you leave, even for short periods.  Dogs that are left outside will dig giant holes in your garden, or tunnel under a fence.  Some dogs will whine, howl or bark constantly beginning at your departure and continuing until you return.  Still others will exhibit a wide range of destructiveness, chewing on anything within their reach.  This can include a cushion, the trim around the door, the curtains or blinds taken right off the windows, the entire sofa, the linoleum floor, electrical cords, his tether and your deck and siding (if left outside), etc.  These behaviours certainly could be a sign of separation anxiety, but they could also indicate something else.

If your dog is exhibiting any of these behaviours, you will first want to explore all the reasons that may explain the questionable behaviour.  The first step in assessing and treating any behaviour problem is always to rule out a medical cause, so visit your veterinarian.  Some other causes of inappropriate elimination could include intestinal parasites, urinary tract infection, kidney problems, boredom, digestive upset, house-training confusion and anxiety.  Inappropriate vocalising, digging and chewing could indicate boredom, digestive upset, parasites, pain, attention-seeking or anxiety.  It is important to recognise that there could be a reason for a dog’s problem behaviours other than separation anxiety.  Here are some questions which can help in your assessment (from Dogs are from Neptune by Jean Donaldson © 1998):

1. Does she seem anxious when you are getting ready to leave her alone?  The tip-offs are restlessness, salivation, pacing, shadowing you around, etc.
2. Is there a prolonged “relief” type greeting when you come home?  Most dogs greet their owners excitedly after absences but dogs with separation anxiety are even more intense.  Often their greeting doesn’t subside and they are still excited minutes later.
3. Does she have a history of vocalising or destructiveness when alone?  Is she a clingy type?
4. Are the accidents only occurring in your absence or is she sneaking any in behind your back when you are home?  If she is also making mistakes when you are at home, this would point towards a health problem or housetraining regression, not separation anxiety.”

The next thing to consider is the dog’s diet, as some nervous behaviours could be caused or intensified by poor nutrition or low-quality ingredients.  For dogs with behavioural issues, a super-premium food with no non-digestible ingredients and no chemical preservatives is recommended.  Easy digestion of quality food can only help your dog’s behaviour.

According to Dr. Lore I. Haug, DVM, MS, DACVB, CPDT, CABC, specialist in Companion Animal Behaviour with the Dept. of Small Animal Medicine and Surgery at Texas A&M College of Veterinary Medicine, there are primarily three distinct manifestations which are commonly referred to as separation anxiety.  “So all these dogs have separation related distress, which I then break down into several subcategories: true pathologic separation anxiety, isolation distress and frustration-induced distress.”  True separation anxiety is when the behaviour occurs anytime the dog’s owner is absent (so the behaviour is not better if someone else is home with the dog).  Isolation distress in much more common, and refers to the behaviour occurring when the dog is totally alone.  Frustration-induced distress has more to do with the environment the dog is kept in than the fact that he is separated from anyone.  Recognising which category your dog falls into can help you better understand his sensitivities and work to ease his distress.

Once other causes have been eliminated and you have identified your dog’s individual source of stress, there are exercises which can help to manage and modify the dog’s behaviour.  The following are just some tips and tricks which have worked for others, please recognise that each dog is an individual and what works for one may not work for others.  If your dog is experiencing separation distress of any kind, consult a behaviour consultant for help in developing a personalised program for your dog.  For a behaviour consultant in your area, visit: www.apbc.org.uk /  http://www.iaabc.org / www.animalbehavior.org.  If you think that your dog may need medication to ease his anxiety, you will want to find a veterinary behaviour consultant who is familiar with the different anti-anxiety drugs, as well as behaviour modification programs.  For a listing of Board Certified Veterinary Behaviourists, visit: www.dacvb.org.

Behaviour Modification Exercise

• Whenever possible, prevent the bad behaviour.  This may mean using baby gates, a crate, or even utilizing a doggie daycare or pet-sitter.  When it does happen, bring as little attention as possible to the undesirable behaviour.

• Make sure your dog gets enough vigorous exercise and has some good stimulating toys when he is left.  One of the best toys out there is the Kong and I recommend that you have three or four of them.  The trick is to stuff the Kong with peanut butter, bananas, yogurt, cottage cheese, canned dog food, etc. and then you FREEZE it.  You will give your dog a stuffed, frozen Kong before you leave, and possibly leave one or two more around the house so he can find them and stay busy.  The reason you freeze it is so it takes an hour or so to get all the stuff out.  By the time he is done with all that licking, he’ll be exhausted!  Associating your departure with something wonderful like this is called counter-conditioning.

• Systematic Desensitisation to being alone.  You need to teach your dog that there is no reason to be destructive when he is left alone.  Remember to leave a Kong or two with your dog when you leave.  If you are confining him in a room or a crate, you will need to get him used to the room or crate while you are home.  For example, if he only goes to the crate when you leave, then it doesn’t feel like part of the home.  Spend time in his safe-place playing with him, feed him there; put him there for a few minutes several times each day when you won’t actually leave the house.  After a week or two, you can put him in there and just go outside and walk around the house and come back in.  The next week start the car, but don’t leave.  Then drive around the block.  We’re gradually building up to him being left alone.  You will also want to practice your ‘getting ready to leave’ routine many, many times each day when you don’t actually leave.  Grab your keys, put on your coat; turn on the radio…  what ever you do to get ready to leave, except you won’t be leaving.  Then, after a week or two you’ll go out the door, and come right back in.  Then you’ll walk around the house; then start the car… move slowly to avoid regression.

• Do not allow yourself to have big emotional good-byes when you leave.  Almost as important is to have a rather low-key greeting when you return as well.  Just slip out the door and slip back in.  Come all the way into the house, take off your coat, and THEN say hello to your dog.  Try counting to 100, that way you’ll have something else to focus on.  He should also earn your attention at other times as well.  If you notice he is following you around the house, re-direct him to do something else (like lie down, or go get a toy) for a bit.  Basically, encourage him to be a bit more independent when you are home, so he won’t feel so lost when you are gone.

• Consistency.  Everyone needs to be on the same page.  The DOG needs to learn that he is part of a human family and we need to teach him what the social rules are in the human world.  Dogs adapt, that’s what we bred them for.  When guessing what he may be feeling and/or thinking, remember that he does not possess some common human emotions (like anger or spite).  Do not put human traits on him, that is a lot to live up to, and it really sets him up for a failure.

Once again, there are more components to a modification program which depend greatly on each particular dog.  Seek the help of a behaviour consultant to help develop a program for your unique situation.

Andrew Luescher, DVM, on Cesar Millan

Human-Animal Bonds: Insight Into Thinking And Behaviour

“Animals notice the details that people don’t notice”

Mississippi State University College of Veterinary Medicine hosted a Human-Animal Bond Lecture Series October 19-23. The focus was to explore all facets of the human-animal bond and raise awareness that the bond extends beyond pets and companion animals to production animals and wildlife. Temple Grandin, Professor of Animal Science at Colorado State University was the keynote speaker Tuesday evening. She also gave a lecture earlier in the day on her experiences with Autism Spectrum Disorder.

Grandin’s keynote address was “How Animals Think and Feel.” Because of her autism, Grandin thinks in pictures, a characteristic that gives her valuable insight into animals’ thinking processes and behaviour. Grandin outlined three thinking styles: (1) oral, visualistic. This is also associative thinking, the way a “search engine” works. (2) Pattern thinking, which is more abstract than a visual thinker. (3) Pure word thinker. “I am a total visual thinker,” Grandin said. “Animal thinking is sensory based, whereas human thinking is language based. Their memories are going to be in pictures/sight, smells, sounds, touch and taste. They are very sensitive to tone of voice. There’s a whole world of cognition without having language.”

Do animals have cognition and thinking ability? They do! “Cognition is the ability to solve problems under novel conditions. It is not operant conditioning and is not hardwired instinctual behaviour,” she explained.

In her lecture and her book Animals Make Us Human, she explained the emotional systems of animals. She wrote, “All animals and people have the same core emotional systems.” She identified the seven basic emotions, which neuroscience has mapped the circuits in the brain: seeking, rage, fear, panic, lust, care, and play.

Seeking is “the basic impulse to search, investigate, and make sense of the environment.” Seeking is a very pleasurable emotion, which includes wanting, curiosity, looking forward, novelty. “Seeking might be a kind of master emotion,” she wrote. Rage might have evolved from the experience of being captured and held immobile by a predator. “We should assume that some captive animals feel frustrated being locked up inside enclosures, barns, apartments, houses, yards and cages, because being locked up is a form of restraint no matter how nice the environment is.”

Grandin addressed the unwanted behaviour that humans create in their interactions with animals, and how to change that behaviour. When you are trouble-shooting the causes of the unwanted behaviour, she emphasized, “Details matter!” Pay attention to such details as: “Where does the behaviour occur? To what or whom is it directed? What does the animal’s posture look like? When does the behaviour occur? What exactly does the animal do? What is the breed or genetic line of the animal?”

A key to understanding animal behaviour is to think in details. “Animals notice the details that people don’t notice,” she explained. Pay attention to “distractions, like shadows, flapping tape, chains, etc. She observed that “calm animals are easier to handle,” so notice the details that are upsetting the animal and you will soon find the cause of the unwanted behaviour.

Rapid movement is a main behaviour stimulus for all kinds of animals. It evokes the seeking response in predatory animals, like making dogs want to chase. It evokes the flight response in prey animals like horses and cattle. The main senses through which most animals react are first, vision, and second, auditory. The dog lives in a “smell detail” world, she explained. This sensory-based concept is important: it is the way animals categorize, for example, into things you can chase and things you cannot chase. In humans, language covers up sensory-based thinking.

Motivators for behaviour are fear (never punish fear), aggression, food reward, social reward, pain motivated, curiosity motivated, reproductive, prey drive, mixed motivation. Two basic types of behaviour are learned and instinctual. So, it is important to figure out what motivates specific behaviours.

The main thing people need to do with horses it to prevent behaviour problems. “Don’t rough them up,” she said. When humans do rough things to horses, they don’t forget and they start to associate specific things with fear. Horses tune into visual or auditory things when something bad happens to them. “The horse is very specific in his/her thinking. You have to figure out what the fear memory is and then you can get rid of it maybe.” For examples, “if a snaffle bit causes pain, and the horse goes berserk when you use a snaffle, then you can substitute another kind of bit, a plain bit. If a man roughs them up, then tend to associate fear with all men. Or if it happens in a specific type of arena, they develop a fear of that type of arena and then generalize. You can work on desensitizing, but if the fear is of men or men with black hats, you can’t get rid of all those. You can desensitize to a point, but fear has a nasty habit of coming back and you can’t completely get rid of it.”

She supports natural horsemanship type training. “Get away from the rough stuff because that only causes problems,” she emphasized. “Get rid of the rough training. Don’t tie a horse to a tree and throw things at them, while they are going berserk and pulling backwards. Rough sacking only creates fear. It especially wrecks Arabs and flighty, high strung horses.” Another example she gave: if the horse bucks when you change gaits, that’s a sign you’ve trained too fast. The saddle feels different at different gaits.”

There are three kinds of stress: fear, pain, and physical stress. Fear is the main emotion in animals. When animals voluntarily cooperate in doing something, they have low levels of stress. When you force an animal to do something, this creates high levels of stress. Also, animals stay calmer if they have companions, especially herd animals.

Grandin says that an animal’s first experience with something new should be a good experience. If the initial experience is adverse, this creates a permanent fear response. Animals then associate the bad experience with something they saw or heard when the bad experience happened. Sometimes even something similar to the bad experience can set off an adverse reaction. They develop specific fear memories related to place (visual), to touch, to sound, or to the feel of something.

With horses and cattle, vision is their main sensor. With dogs, smell is their main sensor, although they also use sound and visual thinking.

Animals are very attuned to tone of voice. Loud, high-pitched sounds hurt animals’ ears. Animals with excitable genetics, i.e., high-strung horses, are the most sensitive. So shouting, yelling, screaming at animals is very stressful to them. Horses “watch” with their ears for potential dangers.

Novelty is both a strong stressor and an attractor. Animals have a see-saw type reaction to new things, much like an emotional switch turning on and off. Seeking can evoke curiosity response, but new things can also evoke fear/flight response. “Animals need to be trained to tolerate a certain amount of change in their routine. Animals need to be habituated to flags, strange people, different vehicles, contrasting colors, etc.” The Seeking emotion turns off fear.

Extreme panic behaviour occurs when animals are suddenly confronted with something new. It is best to avoid putting animals into a hyper-anxiety situation, because fear memories are very hard to get rid of.

When training animals, consider their short attention span. Do intense training for about 15 minutes, then do something fun n have a play session. Always end your training session on a good note. Keep it short! Don’t drag out long training sessions. They are far less effective than short, 15-minute sessions interspersed with play, relaxation. The key to effective training is to find out what emotional signal is driving the behaviour.

Why We Learn More From Our Successes Than Our Failures

If you've ever felt doomed to repeat your mistakes, researchers at MIT's Picower Institute for Learning and Memory may have explained why: Brain cells may only learn from experience when we do something right and not when we fail.

In the July 30 issue of the journal Neuron, Earl K. Miller, the Picower Professor of Neuroscience, and MIT colleagues Mark Histed and Anitha Pasupathy have created for the first time a unique snapshot of the learning process that shows how single cells change their responses in real time as a result of information about what is the right action and what is the wrong one.

"We have shown that brain cells keep track of whether recent behaviors were successful or not," Miller said. Furthermore, when a behavior was successful, cells became more finely tuned to what the animal was learning. After a failure, there was little or no change in the brain - nor was there any improvement in behavior.

The study sheds light on the neural mechanisms linking environmental feedback to neural plasticity - the brain's ability to change in response to experience. It has implications for understanding how we learn, and understanding and treating learning disorders.

Rewarding success

Monkeys were given the task of looking at two alternating images on a computer screen. For one picture, the animal was rewarded when it shifted its gaze to the right; for another picture it was supposed to look left. The monkeys used trial and error to figure out which images cued which movements.

The researchers found that whether the animals' answers were right or wrong, signals within certain parts of their brains "resonated" with the repercussions of their answers for several seconds. The neural activity following a correct answer and a reward helped the monkeys do better on the trial that popped up a few seconds later.

"If the monkey just got a correct answer, a signal lingered in its brain that said, 'You did the right thing.' Right after a correct answer, neurons processed information more sharply and effectively, and the monkey was more likely to get the next answer correct as well," Miller said, "But after an error there was no improvement. In other words, only after successes, not failures, did brain processing and the monkeys' behavior improve."

Split-second influence

The prefrontal cortex orchestrates thoughts and actions in accordance with internal goals while the basal ganglia are associated with motor control, cognition and emotions. This work shows that these two brain areas, long suspected to play key roles in learning and memory, have full information available to them to do all the neural computations necessary for learning.

The prefrontal cortex and basal ganglia, extensively connected with each other and with the rest of the brain, are thought to help us learn abstract associations by generating brief neural signals when a response is correct or incorrect. But researchers never understood how this transient activity, which fades in less than a second, influenced actions that occurred later.

In this study, the researchers found activity in many neurons within both brain regions that reflected the delivery or withholding of a reward lasted for several seconds, until the next trial. Single neurons in both areas conveyed strong, sustained outcome information for four to six seconds, spanning the entire time frame between trials.

Response selectivity was stronger on a given trial if the previous trial had been rewarded and weaker if the previous trial was an error. This occurred whether the animal was just learning the association or was already good at it.

After a correct response, the electrical impulses coming from neurons in each of the brain areas was more robust and conveyed more information. "The signal-to-noise ratio improved in both brain regions," Miller said. "The heightened response led to them being more likely to get the next trial correct, too. This explains on a neural level why we seem to learn more from our successes than our failures."

In addition to Miller, authors include former MIT graduate student Mark H. Histed, now a postdoctoral fellow at Harvard Medical School, and former postdoctoral fellow Anitha Pasupathy, now an assistant professor at the University of Washington.

This work is supported by National Institute of Neurological Disorders and Stroke and the Tourette's Syndrome Association.