Model validation Directory UMM :Data Elmu:jurnal:E:Ecological Economics:Vol32.Issue1.Jan2000:

Table 1 Comparison of model estimates of landed catch and recorded landings tonnes 1991 Species 1992 1993 TAC model logbook model logbook TAC model TAC logbook 68 Blue eye trevalla 125 125 57 125 125 94 243 1533 2215 3000 Flathead 1516 2064 3000 1635 1298 Gemfish a 800 800 576 500 500 586 300 300 354 3196 3664 5000 2249 3184 3232 5000 5000 Blue grenadier 400 166 240 John dory 240 99 240 240 103 782 800 800 613 780 Ling 780 728 780 780 224 246 700 Mirror dory 302 196 800 286 284 1673 1095 1550 1550 789 1500 1500 Morwong 709 250 212 300 192 175 Ocean perch 300 221 192 300 300 334 500 500 173 500 500 Royal red prawns 184 966 1598 620 614 892 600 592 Redfish 785 22 650 22 650 Orange roughy 18 250 22 431 18 197 16 687 13 000 13 000 10 481 500 319 160 152 195 500 Silver trevally 198 182 2119 2694 Warehou 2211 2741 3000 1587 2159 3000 Whiting 1642 2100 1320 777 1471 2000 1091 1331 a Both eastern and western gemfish combined. substantial overestimate of the catch. Restricting effort ensures that, in the model, a variety of species are caught in each month and reduces the potential for catches to be substantially overestimated. Given this, additional constraints were added to the model to restrict the maximum and minimum number of days that can be fished for any one group. These constraints are given by DAYS k, m, g ] n k for V g, k 12 DAYS k, m, g 5 0.3d k 13 for k = inshore sector and Vg, k 0. DAYS k, m, g = 0 for Vg, k = 0 14 where n k is the minimum number of days fished by a boat in sector k on any one group in any month. Eq. 13 applies to boats in the inshore sector only, and is imposed to prevent concentration of effort on any one species groups.

5. Model validation

The process of model verification and validation is not straightforward. Oreskes et al. 1994 claim that verification and validation of numerical mod- els of natural systems is impossible. They argue that the existence of uncertain parameters in a model ensure that it can never be verified as a true representation of the system. While the ability of a model to replicate known events is often used to validate models, this does not prove that the model accurately represents the system Oreskes et al., 1994. Nevertheless, the converse would hold: if a model cannot reasonably replicate known out- comes, then it definitely does not represent the system. To test the ability of the model to replicate actual behavior, the model was used to estimate the landings of each quota species over the period 1991 – 93, and these were compared with actual landings. The model estimates were derived given the actual prices received each month in each year, fuel and other variable costs in each year, and restrictions imposed by the TACs in each year. Prices for each species in each year were obtained from the Sydney and Melbourne markets. Fuel costs were estimated on a cost per day basis, and derived from logbook and survey data ABARE, 1993b. Crew costs and marketing costs were esti- mated as a percentage of the total revenue in each year and were derived from surveys of the fishery over the time period examined ABARE, 1993b. Table 2 Comparison of model and survey estimates of key economic variables m 1990–91 survey a 1992 model 1991–92 survey a 1993 model 1992–93 survey a Variable 1991 model Revenue Inshore trawlers 26.4 25.3 31.3 23.6 28.8 23.3 50.5 60.4 45.2 62.3 40.5 Offshore trawlers 66.0 5.2 2.9 4.0 3.0 3.1 Danish seiners 3.3 Trip costs 13.4 17.2 14.2 Inshore trawlers 16.2 15.9 13.1 24.5 32.5 18.5 36.1 33.7 Offshore trawlers 19.2 1.9 Danish seiners 2.9 1.7 2.2 1.8 1.9 Gross margins 11.9 14.1 9.4 10.5 12.6 Inshore trawlers 10.2 26.0 27.9 26.7 Offshore trawlers 28.6 29.8 21.3 2.3 1.2 1.8 1.1 1.3 Danish seiners 1.4 a Source: ABARE, 1993b. The estimates of landings derived from the model were compared with recorded landings for each year Table 1. ITQs were introduced on a broad basis in 1992, but TACs for orange roughy and gemfish both eastern and western combined were in place in 1991. It was found that limits on landings were also necessary on ling and royal red prawns to prevent overestimation of landings of these species in the 1991 simulation. 2 For the 1992 and 1993 simulations, both model estimated land- ings and recorded landings were less than the TAC for most species. In the majority of these cases, the estimated landings were closer to the recorded landings than the TAC. The model does appear, however, to consistently overestimate landings of a number of high value but low quantity species. The significance of any divergence between the level of landings estimated using the model and recorded landings is difficult to determine. The newness of the ITQ system, problems in allocation and the lack of an established quota trading market may have resulted in difficulties in quota leasing Pascoe, 1993. This in turn may have resulted in less catch than might otherwise have been taken, even though quota may still have been available. Where quota could not be obtained by operators, catch of some species may have been discarded, resulting in recorded landings being lower than the true catch. An implicit assumption in the model is that quota can be transferred between boats within each sector of the trawl fleet and between sectors with no impediments or transactions costs. The estimates of revenue, costs, and gross margin derived from the model were also compared with estimates derived from an economic survey of the fishery ABARE, 1993b Table 2. In most cases, the model estimates of revenues and gross margins were higher than the survey derived estimates, but were generally of similar orders of magnitude. As one estimate is based on a calendar year January – December and the other on a financial year July – June, there is no expectation that the esti- mates should be identical. The relatively higher revenues estimated for the offshore sector using the model is largely a result of using market prices for orange roughy. Most orange roughy is sold directly to processors and incurs little or no handling charges once it leaves the boat. As processor prices were generally not available at the level of detail required in the model, it was assumed that the market price would be similar to the processor price 2 The overestimation of ling and royal red prawns was thought to be because of the catch rates of these species being too high in the model. These are relatively minor species in the fishery and may not have been recorded correctly in the logbook data used in the analysis. Rather than adjust the parameters, constraints were placed on their landing. There- fore, the validation process also involved an element of cali- bration, a common process in model development Oreskes et al., 1994. once the marketing charges had been deducted. As the revenue estimate is before marketing charges are deducted, the total revenue is substan- tially higher than the revenue received from pro- cessors. This higher revenue is largely offset by the higher costs, most of which is as a result of the higher marketing charges assumed in the model. As a result, the gross margin estimated using the model is similar to the survey estimate for the offshore sector.

6. Simulations and results