| Abstract | This thesis presents a novel methodology for computational modelling in cognitive science. It emphasizes identifying sets of models and ranges of parameter values which are equivalent to observed real-world cognitive behaviour. This equivalence is determined by a new statistical measure called relativized equivalence, which places an upper bound on the difference between the model and reality, taking into account the sampling confidence intervals. If the models are constrained to be process models, then this equivalence can also be evidence that the models follow the same algorithm as the real cognitive system.
This methodology also provides guidance for the overall process of modelling. When many models are found to be equivalent, the only way to distinguish among them is to include more empirical data. This usually involves new empirical results with different measures than originally considered. When few models (or none at all) are equivalent, this suggests that either the modelling constraints should be relaxed by removing measures or requiring less accuracy, or that new models should be developed. To demonstrate the usefulness of this methodology, it is applied to a variety of cognitive modelling tasks. The RELACS model of repeated binary choice (Erev & Barron, 2005) is reinvestigated, showing that it is only a good model for situations with simple reward structures. In the original, standard approach, these situations were combined with many more complex situations, so this fine-grained conclusion could not be reached. Furthermore, this analysis shows that an internal component of RELACS is vital to its performance, in contrast with the original conclusion.
The methodology is also applied to a model of peer group interaction, demonstrating how new models can be created, including the use of existing cognitive architectures (ACT-R; Anderson & Lebiere, 1998) to guide and constrain. Finally, situations involving qualitative modelling (i.e., modelling without numerical comparison to real-world data) are explored.
The result is a method for analyzing computational cognitive models emphasizing explanation rather than prediction. This is meant to be usable by any modeller, without requiring extensive mathematical or computational skill. A software toolkit (CCMSuite) is developed to support this process. |