2007

SEA's Methods and Measurement Series

Summaries

Discussion Topic:  Retrospective Pretest

The first brown bag lunch session was Friday, April 27, 12:00-1:30.  The retrospective pretest was the topic discussion at this session held at the Ounce of Prevention Fund. 

Recommended references for this session were:

Pratt, C., McGuigan, W. & Katzev, A.  (2000) Measuring Program Outcomes: Using Retrospective Pretest Methodology. American Journal of Evaluation, 21(3): 341-349.

Lamb, T. (2005)  The Retrospective Pretest: An Imperfect but Useful Tool. The Evaluation Exchange, Volume XI, No. 2, Summer.

Dr. Mary Kay Falconer prepared a retrospective pretest reference sheet for all those attending and has incorporated several points raised during the session.  For more information, please contact Dr. Falconer at  mfalconer@ounce.org or 850-921-4494(ext. 134)  Our thanks to all who attended!

Discussion Topic:  Participant Retention in Program Evaluations

The second brown bag lunch session in our SEA Methods and Measurement Series was held on  Friday, May 25, 12:00-1:30, at the Tallahassee Community College Institute's Conference Room #118 (located on the bottom floor of the Mary Brogan Museum, 300 West Pensacola).

Engaging and retaining participants in a program are typically key components in a program’s implementation.  The achievement of program outcomes is also often linked to a program’s ability to enroll members of its target population and retain those who enroll.  This session will address several questions about participant retention in program evaluations, including:

• Where do measures of retention fit in a LOGIC Model?

• Are measures of retention appropriate in summative or outcome evaluations?

• What are common measures of retention and what are their strengths and weaknesses?

• How can we help programs improve participant retention?

Dr. Mary Kay Falconer and Joanna Arrington prepared a summary of the discussion. Questions regarding this workshop should be directed to Mary Kay Falconer (mfalconer@ounce.org) or Joanna Arrington (jarrington@ounce.org) at the Ounce of Prevention Fund.

Discussion Topic:  Measuring Customer Satisfaction

The third brown bag lunch session in our SEA Methods and Measurement Series was held on  Friday, July 27, 12:00-1:30, at the Agency for Health Care Administration (Building #3, Conference Room C). 

Measuring the satisfaction of those we serve is an evaluation task that is completed for many programs in both the private and public sectors.  Selection of the appropriate survey instrument, protocols for survey administration and data collection, techniques for analyzing responses and displaying results, and selection of the target audience are among several issues that should be considered carefully.

Bill Dahlem and Rachel LaCroix, Agency for Health Care Administration, facilitated this discussion and prepared a handout for the participants.

Discussion Topic:  Web-Based Surveys

The fourth brown bag lunch session in our SEA Methods and Measurement Series was held on  Friday, August 24, 12:00-1:30, in the Tallahassee Room at City Hall (300 South Adams). 

Those interested in listening to as well as sharing approaches for using web-based surveys were encouraged to attend this brown bag discussion.  Participants discussed the most popular web survey tools (free and those with costs).  Participants also discussed tips and techniques to use to increase response rates and frame questions.

Dr. Tamara Bertrand-Jones, Office of Research, Division of Student Affairs, Florida State University led the discussion and prepared a handout.

SEA's Writing Workshop

On September 26, 2007, SEA offered a Writing Workshop taught by Connie Ruggles, Agency for Health Care Administration, and Deanna Hamilton and Kathy McGuire, Office of Program Policy Analysis and Government Accountability.  The class offered participants a good refresher on how to communicate clearly, succinctly, and with punch.  These principles can also help agency staff comply with the Florida Governor's Plain Language Initiative.

Discussion Topic:  Cost-Benefit Analysis

The fifth brown bag lunch session in our SEA Methods and Measurement Series was held on  Friday, October 26, 12:00-1:30, in the Tallahassee Room at City Hall (300 South Adams).

“Cost-benefit analysis (CBA) is considered the preeminent method of economic evaluation because it allows for the direct comparison of different types of outcomes resulting from a variety of actions.  If done correctly, it provides the most comprehensive monetary measures of the positive (beneficial) and negative (costly) consequences of a possible course of action…” (page 127). 

 Messonnier, M. & Meltzer, M. (2003).  Cost-Benefit Analysis.  In Prevention Effectiveness, A Guide to Decision Analysis and Economic Evaluation, 2nd Edition (Eds. Haddix, A.C., Teutsch, S.M., & Phaedra, S.C.), Oxford University Press, New York.

DISCUSSION CO-FACILITATORS:   Gary W. Walby, Ph.D., M.S.P.H., M.S. and Mary Kay Falconer, Ph.D., Research, Evaluation and Systems Unit, Ounce of Prevention Fund of Florida prepared a document entitled "Cost-Benefit - - Technique or Formula".

Discussion Topic:  Highlights from the American Evaluation Association's Annual Conference

Many SEA members attended the recent American Evaluation Association (AEA) Conference in Baltimore, November 7-10.  Those who attended agreeed that the conference had a lot to offer.  Plenary sessions, multi-paper sessions, technique demonstrations, and think tank sessions were all part of the program mix with lively debates about advances in evaluation research.  Participants met at the Ounce of Prevention Fund of Florida on Friday, November 30th, to share and hear about experiences at the AEA conference.