Report on the 2003 South East Data, Assessment, and Review (SEDAR) Workshop
to Review the Assessments of the Status of the Stocks of Vermilion Snapper and
Black Sea Bass from the South east of the U.S.
By
Jon Helge Vølstad1, Ph.D.
Versar, Inc.
9200 Rumsey Road
Columbia, Maryland 21045
USA
1Representing Center of Independent Experts, Operated by the Center Institute of
Cooperative Institute for Marine and Atmospheric Science (CIMAS)
University of Miami
Executive Summary
The SEDAR II panel review workshop on vermilion snapper and black seabass
assessments was competently chaired, and conducted in a spirit of cooperation and
teamwork. The assessments, conducted by outstanding stock assessment biologists, were
subject to a rigorous and very open peer review process that identified the most likely
sources of uncertainty. It was agreed that the assessments were based on appropriate
assessment models and used the best available data. However, several potential sources
of bias and uncertainty in these data were identified during the review. Uncertainty in the
stock assessments relate to the extensive dependence on fisheries-dependent indices of
abundance, incomplete spatial coverage, and poor information about discards. Improved
monitoring of the stocks will require adequate data on discards from all fishery segments.
The assessment of vermilion snapper was appropriately based on a forward-projecting
length-structured model because of limited age sampling of the catches for this species,
and bias in available data on age composition from fisheries-dependent samples.
Assessment results for this species are uncertain, but indicate that overfishing is
occurring but that the stock probably is not overfished now. There is major uncertainty in
determining whether or not the stock is overfished because no reliable functional stockrecruitment
relationship could be established based on available data. In addition, the
estimated abundance indices used in the assessment of vermilion snapper are based on a
limited spatial coverage that does not fully reflect the entire stock.
The stock assessment of black seabass was based on an age-structured forward projection
model. Results based upon the best available data used in the assessment documents that
overfishing is occurring and that the stock is overfished. The spatial coverage of survey
data for this species was substantially better than for vermilion snapper. It is
recommended that fishery independent sampling be expanded to improve the reliability
of stock assessments for both stocks. In addition, improved assessments and monitoring
of stock status will require more and improved data on discards.
1. Background
The South East Data, Assessment, and Review (SEDAR) process is a new program that is
part of the NMFS- Southeast Fisheries Science Center’s program for quality control and
assurance of stock assessments in the South East region. The SEDAR is a process
conducted by the South Atlantic Fisheries Management Council (SAFMC) in close
coordination with NMFS and the Interstate Commissions to ensure the scientific quality
and credibility of stock assessments, and to assure that they continue to support effective
fishery management. The SEDAR process comprises a Data Workshop, an Assessment
Workshop, and a Stock Assessment Review Workshop conducted in sequence. The
SEDAR II review panel workshop for black seabass (the component of the stock south of
Hatteras, NC) and vermilion snapper stock assessments was held in Raleigh, NC at the
Holiday Inn Brownstone Hotel from February 25 to 28, 2003. I agree with the findings
and recommendations that are detailed in the SEDAR II workshop review panel
consensus and advisory reports. In this report, I evaluate the review process, and briefly
summarize the findings and recommendations, with focus on my experience as a
reviewer on the panel. This report should be read in conjunction with the two reports
prepared by the review panel.
2. Description of Review Activities
The SEDAR Review Workshop to review stock assessment of vermilion snapper and
black seabass was chaired and facilitated by Dr. Norman Hall in a very organized and
effective manner, and was conducted in a spirit of cooperation and teamwork.
Assessment Workshop reports for the two stocks under consideration, vermilion snapper
and black seabass, were made available for review a few days before the meeting. During
the SEDAR II meeting, each stock assessment was presented by the responsible
assessment expert, and reviewed by the panel. The 12-member review panel represented
a broad area of expertise in fisheries, and included participants from the:
• NMFS-Southeast Fisheries Science Center
• NMFS-Northeast Fisheries Science Center
• South Atlantic Fisheries Management Council
• Snapper/Grouper Advisory panel
• Non-Government (NC Environmental Defense)
• Center for Independent Experts (chair and reviewer).
Review activities during the workshop involved panel discussions on assessment validity
and results, and the development of consensus recommendations and conclusions
following the presentation of assessments for each stock. Mr. Greg Waugh, a panel
member from the SAFMC, did an excellent job documenting the consensus review
comments for inclusion in the reports authored by the panel. The reviews focused on the
evaluation of the adequacy and appropriateness of:
• Fishery-dependent and independent data used in the assessment (i.e. was the best
available data used in the assessment);
• Application of models used to assess these species and to estimate population
benchmarks (MSY, Fmsy, Bmsy and MSST, i.e. Sustainable Fisheries Act items);
• Models used for rebuilding analyses.
During the week following the review meeting, the entire panel took part in the
development of the two summary reports by providing input, and by reviewing comments
from fellow panel members. Dr. Norman Hall did an outstanding job leading this
inclusive process.
2.1. Input-Data
The CIE reviewers did not receive the CD documenting the Data Workshop, and thus the
evaluation of the quality of input-data relied entirely on the brief descriptions in the two
stock assessment reports, and verbal information provided by the presenters of the stock
assessments and by support staff and other attendees. The available information was not
sufficient for a comprehensive review. The panel focused on the accuracy and reliability
of input-data, and sought information about the availability of additional data that
potentially could be used to enhance the stock assessments. Receiving special attention
were potential effects related to gear catching efficiency and selectivity, and the spatial
and temporal coverage of fisheries-dependent and fisheries-independent (i.e., MARMAP)
data used to derive abundance indices and to estimate catch and its characteristics over
time.
2.2. Assessment and Projection Models
The models and their specifications were only evaluated in general terms because the
technical descriptions of the model structures provided in the assessment reports were
sketchy and insufficiently complete for a thorough review. The Review Panel relied
heavily on the information provided in the verbal presentations. The appropriateness of
the models was evaluated by taking into account the life history and type of data
available for each species. The evaluation of projections focused on the likelihood and
range of input parameters applied.
3. Summary of Findings
The panel documented its review findings in a Peer Review Panel Consensus Report that
includes detailed comments on the individual species assessments and the Panel's
findings on the status of the stock and the fishery. The panel also co-authored a Summary
Stock Status Report in support of the Fisheries Management Council. I agree with these
findings and recommendations, which also incorporated all my input. In the following, I
will add some comment about the review process.
In my opinion, this second SEDAR review process clearly supports the Council’s
objective to continually improve the quality of stock assessments and their relevance to
support sound fishery management. The review process was open, and the assessment
scientists from the agencies did a great job presenting the assessments to the panel. The
panel members had broad and complimentary expertise that covered all the review
subjects. The panel greatly benefited from the input from the meeting support staff and
other attendees, throughout the review process.
One criticism I have is that the two stock assessment reports that formed the basis for the
review provided limited details on the input-data and model specification. I recognize
that the stock assessment scientists responsible for the Assessment Workshop reports
may have had insufficient time to fully document the methods. However, due to this lack
of documentation, the Review Panel was limited to base much of their evaluation on the
information provided in the verbal presentations.
It is possible that the detailed descriptions sought by members of the Review Panel are
presented in the reports of the Data workshop. However, this information was not made
available for the review panel meeting, but should have been.
The data collections to estimate the characteristics of commercial catches were not
sufficiently documented to evaluate if catches from different spatial or temporal zones, or
from different fishing sectors, have been representatively sampled. Also, information on
the sampling intensity by fishing sectors, and the method for combining various catch
samples across sectors, is insufficient to evaluate their adequacy and appropriateness.
4. Conclusions and Recommendations
The NMFS assessment scientists and supporting staff did an outstanding job presenting
the assessment results, and were very helpful throughout the review meeting by
answering questions related to the panel's interpretation of the available data and results.
The effectiveness of the review process was substantially enhanced by the contributions
from the Assessment Workshop/Review Panel Support Staff and from the South Atlantic
Fisheries Management Council Staff and sub-committee members. In most cases, this
diverse group of fisheries experts could clarify issues related to assessment models and
the available input-data. Although the descriptions in the assessment reports of the model
specification and methods used to collect and to analyze the data used in the assessments
were not sufficiently complete for a thorough and comprehensive technical review, I feel
that the stock assessments were based on suitable methods and the best available data.
I support the conclusions and recommendations presented by the review panel in the
Second SEDAR assessment consensus report, and will only highlight a few issues here.
I strongly recommended that the assessment reports for future stock assessments include
more detailed descriptions of the methods of data collection, analysis, and the use of
these data for stock assessment. It is recommended that the assessment reports for future
stock assessments include detailed descriptions of the methods of data collection,
analysis, and the use of these data for stock assessment. Sufficient details of the methods
of data collection should be provided to allow the Review Panel to assess the extent to
which catches from different spatial or temporal zones or from different fishing sectors
have been representatively sampled, how the various samples are combined, and the
sampling intensity that has been applied to the different sectors. Minimum levels of
sampling intensity and spatio-temporal coverage to achieve acceptable precision for key
population parameters should be specified by during the Data and Assessment
Workshops, and those sample sizes should be increased if the sampling intensity should
fall below this minimum level. The sampling designs of the various data collection
methods should be reviewed for statistical adequacy (sampling intensity and spatiotemporal
coverage). It is possible that this was addressed in the Data Workshop. If so, I
recommend that this also be summarized in the assessment workshop reports for
completeness.
Abundance indices and estimates of population characteristics from fisheries-dependent
data currently provide essential information for the assessments of Vermillion snapper
and black seabass. Commercial catch-per-unit-effort (CPUE) statistics should be used
cautiously to track changes in the stock over time. Fishermen often have the ability to
locate areas of high local abundance even when overall stock size is low, and concentrate
their fishing effort there. The fisheries literature contains substantial evidence that
fishery-dependent indices of abundance can at times underestimate the degree of decline
in a stock because they do not follow a simple linear relationship with stock size. By
targeting local concentrations (patches) of fish that they find based on their expert
knowledge, fishers can often maintain a relatively high catch per unit effort even when
the overall abundance is in decline. This is especially the case for species that aggregate
in structured habitats (e.g., reef fish), or schooling fish that can be located by
sophisticated acoustic fish finding equipment. This is one major reason that CPUE often
fail to track the true status of the stock for wide variety of fisheries, as documented by
Gunderson (1994) and numerous references therein. Ulltang (1996) shows dicrepancy
between VPA and fisheries-independent abundance indices from trawl and acoustic
surveys. Pennington and Strømme (1998) discuss the case of Newfoundland Cod, which
is one of the gravest examples, and show how CPUE from the commercial fishery
indicated a stable stock while the true abundance was declining towards a collapse (the
fisheries-independent abundance indices from trawl surveys showed a declining trend
during the same period). This has also been observed for logbook data (Baum et al.
2003).
Well-designed fisheries-independent surveys tend to track trends in fish abundance more
accurately because they sample habitats and density levels in proportion to their aerial
extent. For such reasons, the fisheries-independent data should receive higher weighting
as the time series increases. I strongly agree with the panel’s proposal that MARMAP
conduct a synoptic study of their gear to provide a basis for comparing relative gear
efficiencies. This would allow a long time series of fishery-independent abundance
indices to be developed. Over time, it is strongly recommended that the assessment
assign more weight to fisheries-independent survey indices from the MARMAP program.
MARMAP should also be expanded into deeper water to improve the spatial coverage of
the stock.
Although fisheries-dependent data have limitations with respect to tracking of trends in
abundance, it is recommended that commercial logbook data be evaluated for inclusion
as auxiliary information in stock assessments. Their extended use could help build trust
with the fishing industry, and could potentially improve stock assessments by providing
information about discards, and improving the spatial and temporal coverage of catch
data. The usefulness of incorporating catch data from logbooks could potentially be
evaluated through a pilot study that applied survey sampling to select a representative
sample of logbooks. This could be a cost-effective way to determining whether it is
possible to develop a reliable fishery-dependent index of abundance from such data.
The age-based forward projecting method is particularly sensitive to inaccurate
information on catches at age, for example related to limited sampling coverage (spatially
and temporally) of landings, and unreported discards. If feasible, I recommend that the
variability in assessments caused by sampling variability in estimated landings in number
by age be evaluated, for example by applying bootstrapping to port sampling data in
connection with the model runs. Also, biased assessments (of unknown magnitude) could
occur when multiple survey indices are used for “tuning”, especially if they are assigned
equal weights (during periods of overlap), regardless of spatial coverage and precision.
Such bias can be severe when some surveys only cover a limited fraction of the
distribution area of a species. One way to reduce or eliminate such bias is to combine the
respective survey estimates by using a composite estimator that applied weights that
depend on coverage and precision to each abundance series, and then apply the combined
series in tuning the model. Additional post-stratification might be appropriate when
surveys overlap in sub-area. Examples of the combination of multiple indices are
presented in Korn and Graubard (1999) and Vølstad et al. (2003).
The current stock assessment models for vermilion snapper and black seabass apply a
large number of parameters that are difficult to track. The external analysis of multiple
survey indices of abundance might provide a better understanding of the input data, make
the weighting more transparent, and result in a more parsimonious stock assessment
model.
References
Baum, Julia K., Ransom A. Myers, Daniel G. Kehler, Boris Wrom, Shelton J.
Harley, Penny A. Doherty. 2003. Collapse and conservation of shark populations in the
northwest Atlantic. Science 299: 389-392.
Gunderson, D.R. 1994. Surveys of Fisheries Resources, John Wiley & Sons.
Korn, E.L., and B.I. Graubard. 1999. Analysis of Health Surveys. Wiley, New York.
38299.
Ulltang, Ø 1996. Stock assessment and biological knowledge: can prediction uncertainty
be reduced? ICES Journal of Marine Science 53: 659-675.
Pennington, M. and J.H Vølstad. 1994. The effect of intra-haul correlation and variable
density on estimates of population characteristics from trawl surveys. Biometrics 50:
725-732.
Vølstad, J.H, W.R. Richkus, S. Gaurin, and R. Easton. 1997. Analytical and Statistical
Review of Procedures for Collection and Analysis of Commercial Data Used for
Management and Assessment of Groundfish Stocks in the U.S. Exclusive Economic
Zone Off Alaska. Prepared for the U.S. Department of Commerce, National Marine
Fisheries Service, Alaska Fisheries Science Center, Seattle, Washington. 172 pp.
Vølstad, J.H., N.K. Neerchal, N.E. Roth, and M.T. Southerland. 2003. Combining
biological indicators of wateshed condition from multiple sampling programs – a case
study from Maryland, USA. Ecological Indicators [In Press].
Appendix A: Bibliography of Material Provided
Provided prior to SEDAR panel review workshop:
Report of Black Seabass Stock Assessment Workshop, Second SEDAR Process,
Beaufort, North Carolina, January 6-10, 2003. Prepared for South Atlantic Fishery
Management Council, Charleston, South Carolina, 14 February 2003.
Report of Vermilion Snapper Stock Assessment Workshop, Second SEDAR Process,
Beaufort, North Carolina, January 6-10, 2003. Prepared for the South Atlantic Fishery
Management Council, Charleston, South Carolina. Issued February 13, 2003.
Material provided during the SEDAR panel review workshop included:
McGovern, J.C., M.R. Collins, O. Pashuk, and H.S. Meister. 2002. Temporal and spatial
differences in life history parameters of black sea bass in the southern United States.
North American Journal of Fisheries Management 22: 1151-1163.
Poffenberger, J. 2002. A report on the supplemental discard data for the Southeast
Fisheries Science Center’s Coastal Fisheries Logbook Program. Sustainable Fisheries
Division Contribution No. SFD-02/03-183. (Preliminary Report)
Stock Assessment and Fishery Evaluation Report for the Snapper Grouper Fishery of the
South Atlantic. Volume I. Prepared for South Atlantic Fishery Management Council,
Charleston, South Carolina, November 1999.
Vaughan, D.S., Boxian Zhao, Mark R. Collins, John C. McGovern, and H. Scott Meister.
1988. Evaluation of multiple survey indices in assessment of black seabass from the U.S.
South Atlantic coast. Fisheries Stock Assessment Models. Alaska Sea Grant Program.
AK-SG-98-01.
Additional Material Consulted:
Improving Fish Stock Assessments. National Academy Press. Washington, DC, 1998.
176 pp.
Improving the Collection, Management, and Use of Marine Fisheries Data, 2000.
Ocean Studies Board, National Research Council, 236 pp.
Appendix B:
STATEMENT OF TASK
Subcontract between the University of Miami and Versar, Inc. (Dr. Jon Vølstad)
February 12, 2003
General
The South East Data, Assessment, and Review (SEDAR) process for stock assessment
and review is used in the NMFS- Southeast Fisheries Science Center’s area of
responsibility. This new program provides the framework for independent peer review of
stock assessments undertaken jointly by NMFS-SEFSC, three Regional Fishery
Management Councils, and two Interstate Fishery Commissions, and state fishery
agencies. The SEDAR process uses a three phase approach: a data workshop, an
assessment workshop, and a peer review panel workshop. The peer review panel is
composed of stock assessment experts, other scientists, and representatives of the
Council, the fishing industries, and non-governmental conservation organizations. The
communication elements of SEDAR include a stock assessment report from the
Assessment Workshop, a review panel report evaluating the assessment(s) (drafted
during the Review Panel Workshop), presentation of the peer reviewed assessment results
to the Council(s) and public, and publication of collected documents for stock
assessments considered in that cycle of the SEDAR process.
The assessments to be reviewed by this SEDAR Peer Review Panel are of black seabass
and vermilion snapper from the South Atlantic Fishery Management Council area of
jurisdiction. A data workshop was held October 6–10, 2002 in Charleston, SC. The
assessment workshop was held January 6–10, 2003 in Beaufort, NC. The SEDAR
Review Panel for the black seabass and vermilion snapper assessments will include up to
12 members: 1 senior assessment scientist each from NMFS- NEFSC and -SEFSC, 1
Council Staff scientist and 2 assessment scientist members of the Scientific and Statistical
Committee from the South Atlantic Fishery Management Council, 1 commercial
fisherman from the Snapper-Grouper Advisor Panel (shared by two individuals, each
with special experience in one of the species), 1 scientist representative from a nongovernmental
organization, and 2 members (chair and reviewer) from the Center for
Independent Experts (CIE). Assessment scientists from NMFS-SEFSC will present the
assessments and be available to provide supplemental information as requested by the
review panel.
SEDAR Assessment Review Panel Tasks-
The Panel will evaluate the black seabass and vermilion snapper assessments, the input
data, assessment methods, and model results as put forward in the stock assessment
workshop report.
Specifically, the review panel will:
1. Evaluate the adequacy and appropriateness of fishery-dependent and independent
data used in the assessment ( i.e. was the best available data used in the
assessment)
2. Evaluate the adequacy, appropriateness and application of models used to assess
these species and to estimate population benchmarks (MSY, Fmsy, Bmsy and
MSST, i.e. Sustainable Fisheries Act items);
3. Evaluate the adequacy, appropriateness, and application of models used for
rebuilding analyses;
4. Develop recommendations for future research for improving data collection and
the assessment;
5. Prepare a report summarizing the peer review panel’s evaluation of the black
seabass and vermilion snapper stock assessments. (Drafted during the Review
Workshop, with the Final report due two weeks after the workshop- March 14,
2003);
6. Prepare a summary stock status report including management recommendations.
(Drafted during the Review Workshop, with the Final report due two weeks later -
March 14, 2003.)
It is emphasized that the panel’s primary duty is to review the existing assessment. In the
course of this review, the Chair may request a reasonable number of sensitivity runs,
additional details of the existing assessment, or similar items from technical staff.
However, the review panel is not authorized to conduct an alternative assessment, or to
request an alternative assessment from the technical staff present. To do so would
invalidate the transparency of the SEDAR process. If the review panel finds that the
assessment does not meet the standards outlined in points 1 through 3, above, the panel
shall outline in its report the remedial measures that the panel proposes to rectify those
shortcomings.
The Review Panel Report is a product of the overall Review Panel, and is NOT a CIE
product. The CIE will not review or comment on the Panel’s report, but shall be
provided a courtesy copy, as described below under “Specific Tasks.” The CIE products
to be generated are the Chair’s report, also discussed under Specific Tasks.
Specific Tasks
Designee will serve as a panelist on the SEDAR stock assessment peer Review Panel for
black seabass and vermilion snapper. The panel will convene in Raleigh, NC at the
Holiday Inn Brownstone Hotel during the week of 24 February 2003. The Panel meeting
will begin mid-day on February 25 and conclude early afternoon on February 28, 2003.
The panel will review stock assessments provided for black seabass (stock south of
Hatteras, NC) and vermilion snapper in the area of jurisdiction of the South Atlantic
Fishery Management Council.
The SEFSC shall provide the CIE with copies of the following two documents for
distribution to the Reviewer.
Report of Black Seabass Stock Assessment Workshop, Second SEDAR Process,
Beaufort, North Carolina, January 6-10, 2003. Prepared for South Atlantic
Fishery Management Council, Charleston, South Carolina, 14 February 2003.
Report of Vermilion Snapper Stock Assessment Workshop, Second SEDAR
Process, Beaufort, North Carolina, January 6-10, 2003. Prepared for South
Atlantic Fishery Management Council, Charleston, South Carolina, Issued
February 13, 2003.
It is estimated that the Reviewer’s duties will occupy a maximum of 12 workdays; a
couple of days prior to the meeting for document review; four days at the SEDAR
meeting, and a few days following the meeting to ensure that final review comments on
documents are provided to the Chair and to complete a CIE review report.
Roles and responsibilities:
1. Prior to the meeting panelists will be provided with the stock assessment
workshop report and other associated documents on the black seabass and
vermilion snapper. All panelists shall read these documents to gain an in-depth
understanding of the stock assessment and the resources and information
considered in the assessment;
2. During the review panel meeting, participate, as a peer, in panel discussions on
assessment validity, results, recommendations, and conclusions. Participate in the
development of the Peer Review Panel Report and Summary Stock Status Report;
3. Review and provide comments to the Panel Chair on the Draft Peer Review Panel
Report and Summary Stock Status Report;
4. No later than March 14, 2003, submit a written report1 consisting of the findings,
analysis, and conclusions, addressed to the “University of Miami Independent
System for Peer Review,” and sent to Dr. David Sampson, via email to
David.Sampson@oregonstate.edu, and to Mr. Manoj Shivlani, via email to
mshivlani@rsmas.miami.edu.
1 The written report will undergo an internal CIE review before it is considered final. After completion, the
CIE will create a PDF version of the written report that will be submitted to NMFS and the consultant.
Contact persons:
NMFS contact: Dr. John Merriner, Beaufort Laboratory, 101 Pivers Island Road,
Beaufort NC 28516 Phone 252-728-8708, FAX 252-728-8784, E-mail
john.merriner@noaa.gov.
SAFMC contact: Mr. Gregg Waugh, One Southpark Circle, Suite 306, Charleston, SC
29407, Phone 843-571-4366, FAX 843-769-4520, E-mail gregg.waugh@safmc.net.
ANNEX I: REPORT GENERATION AND PROCEDURAL ITEMS
1. The report should be prefaced with an executive summary of findings and/or
recommendations.
2. The main body of the report should consist of a background, description of review
activities, summary of findings, conclusions/recommendations, and references.
3. The report should also include as separate appendices the bibliography of all
materials provided and a copy of the statement of work.
Please refer to the following website for additional information on report generation:
http://www.rsmas.miami.edu/groups/cie.