Decision making in dynamic environments: The effects of instructions inducing an internal task representation, and of outcome feedback
Human information processing in dynamic decision environments is relatively unexplored. These are environments where decisions are made continually and modified on the basis of feedback, as well as complicated by the occurrence of time lags. The present study formulates a general model of such decision making phrased in terms of Feedback, Filtering, Situation Assessment, Internal Model of the decision environment. Decision Planning, and Decision Implementation. The empirical part consisted of an experiment using a computerized management game which was played individually. Participants were expected to make nine sets of decisions over a series of nine decision rounds. Each set was fed into the simulation as a result of which a new, updated, decision environment was created for the participants.
Independent variables were amount of instructions allowing the formation of an internal model of the decision environment (Extensive versus Normal Instructions), frequency of outcome feedback (every round versus every other round), and time-on-task (number of rounds played). Dependent variables were decision performance (measured by retained earnings), accuracy of the subjects' performance prediction, accuracy of the internal model (measured by a knowledge test), and the degree to which the eight decision variables were utilized (changed) by the subjects. Besides, several aspects of the subjects' query behavior were studied, including number of historic queries, number of redundant queries, and the number of query links corresponding to first-order Markov chains.
It was hypothesized that with the more extensive instructions and with frequently presented outcome feedback subjects would show better decision performance, higher internal model accuracy, and higher prediction accuracy. Moreover, the number of historic queries was expected to decrease and the number of relevant queries and the number of first-order query links to increase under the same conditions. The same trends were predicted to take place over time. Finally, Extensive Instructions and Frequent Outcome Feedback were predicted to result in increased usage of the decision variables.
Results showed that amount of instructions had a negative effect on performance and prediction accuracy (lower with Extensive Instructions), although the effect on prediction accuracy was nonsignificant. Internal model accuracy, on the other hand, was affected in the expected direction (higher accuracy with Extensive Instructions) by this independent variable. Frequency of outcome feedback had a negative effect on performance and prediction accuracy (lower with Frequent Outcome Feedback). For the performance data this effect was mainly due to subjects in the Infrequent Feedback group making higher earnings on rounds where outcome feedback was made available to them. On the alternate rounds there was no difference in earnings between the two Feedback groups. The effect of the feedback manipulations on internal model accuracy was nonsignificant.
Of the query behavior measures the number of historic queries showed expected effects of both amount of instructions and frequency of outcome feedback. The effects on the other query behavior measures showed an inconsistent pattern. The extent to which the decision variables were used changed in the expected direction as a result of Extensive Instructions and Frequent Outcome Feedback, but only significantly so for two and one decision variables, respectively.
Finally, learning effects were observed only for performance and prediction accuracy.
The experimental design model of this study utilized four covariates to correct for experimental groups having different means on these covariates and to correct for correlations between these covariates and the dependent variables. These correlations were quite large, boosting the overall model R-squared value in some cases even as high as 0.60. This finding was taken as a sign of the importance of individual differences, i.e., background variables such as experience and age, in explanations of decision performance and behavior. An in-depth analysis of these individual differences also showed how background variables moderated the effects that were of primary interest to this study, namely, those of amount of instructions and frequency of outcome feedback.
The discussion focused on the phenomena of cognitive overload and the stability of people's internal models (even if they are in error) as an explanation for some of the paradoxical findings of this study. The results were also discussed in relation to methodological problems that probably were inherent to this study and in relation to their implications for decision support and future research.