April 30, 2009 - EMS Online Collaborative Web Site Workshop

Evaluation Monitoring System Collaborative Web Site Training Workshop.

We focused on how to calculate the excess medical cost saved by each grantee.

Then we demonstrated how to post the bar charts, pictures, text, and related information on the Collaborative Web Site.  This web site allows each of us to create and update our own web page on our shared site.  The workshop was a useful training opportunity for new grantees/evaluators and a useful refresher for those who already generate the Evaluation Monitoring System information and use the collaborative site.  (The first page summarizes our collective data and can be seen in the top left corner.  Collectively we have saved the State of Arkansas over $94 million in excess medical costs (by helping people stop using tobacco).

Below is the Contact Information page of our collaborative web site, highlighting the individual accomplishments of our grantees (as compared with our collective data in the top left corner of this blog).

Dr. Calvin Johnson, Dean of the School of Education at the University of Arkansas at Pine Bluff, welcomed the grantees and evaluators at the workshop (see below).

Evaluators and grantees contemplated the potential of this site and the power of combining our data instead of reporting only by each individual grantee.

We had a guest speaker during the workshop training.  Oluwakemi Talabi (Kemi) is a Tobacco Cessation Program Analyst from the Arkansas Department of Health.  She provided us with an update on the type of information we could expect to receive from the new vendors operating the Quitline.  It was impressive and and excellent opportunity to link up with the Quitline contact people in the Tobacco Prevention and Cessation Program in the Arkansas Department of Health.  (See the contact information cards below.)

This is Kemi speaking to the group (below).  She was gracious and informative.

The Clearing the Air in Communities of Color Conference was held concurrently with the evaluation training workshop.  We were all able to attend parts of the conference including the skits (see below) which were very effective tools to reach minority youth.

After the skits concluded, many of the grantees and evaluators reconvened and discussed next steps with Dr. Fetterman.  Dr. Fetterman explained that he would be providing his comments on each of the mid-year reports (which he has already completed).  In addition, he would continue to provide additional technicals assistance until the Evaluation Monitoring System and Visual Alert System were used consistently by each grantee.  Dr. Fetterman also agreed to continue to provide additional one-on-one assistance concerning posting on the Collaborative Web Site (so that the information would be up-to-date and visible to the public).

Summer Evaluation Workshop

Evaluation Workshop. The free summer evaluation workshop was considered a tremendous success, according to participants, workshop facilitators, and legislative officials. Inquiries came as from as Ethiopia and Liberia. However, most of the interest came from Arkansas and the mid-west. There were approximately 100 participants.

The workshop began with welcome messages from University of Arkansas at Pine Bluff academic officials, including Drs. Johnson and Benjamin. Mayor Redus also welcomed workshop participants.

Dr. Fetterman provided an outline of the workshop, including plenary speakers and workshop facilitators. He also highlighted the local Arkansas evaluation group's meeting during the workshop.

Plenary speakers included: Senator Tracey Steele, Representative Flowers, Kevin Collins (Centers for Disease Control), Charles Gasper, Missouri Foundation for Health, and Charlotte Lewellen-Williams (University of Arkansas Clinton School of Public Service).

The House of Representatives presented a Citation for our work during the workshop:

The topics and speakers were as follows:

• Dr. Stewart Donaldson - Practical Program Evaluation
• Dr. Tarek Azzam - Technological Tools
• Dr. David Fetterman - Empowerment Evaluation
• Dr. Rosalie Torres - Interviewing Skills
• Dr. Abraham Wandersman - Getting to Outcomes

The Arkansas Group of Evaluators held its inaugural meeting during the workshop (at night). The chair, Linda Delaney, orchestrated the evening meeting and plans to continue to build this organization (an official local affiliate of the American Evaluation Association).

Center for Disease Control and Prevention literature was made available to participants free of charge, including:

• Best Practices for Comprehensive Tobacco Control Programs
• Introduction to Process Evaluation in Tobacco Use Prevention and Control
• Introduction to Program Evaluation for Comprehensive Tobacco Control Programs
• Pathways to Freedom: Winning the Fight Against Tobacco

A detailed portrait of the workshop is provided at the Arkansas Evaluation Center blog.

MISRGO Annual Evaluation Report 2008

Report. The Annual Report provides a brief summary of evaluation activities and accomplishments throughout the year. The most significant achievement by the group is presented in their Evaluation Monitoring System. To-date, the group has save the state $94,944,281.96 in excess medical costs. This is an increase of approximately $10,188,113.56 since last year. In addition, the Visual Alert System is being implemented this year. It involves the use of bar charts to graphically illustrate what kinds of activities the grantees are engaged in, as well as the frequency of these activities - across grantees. Additional accomplishments include workshop training, blogs, and publications. Plans for the Summer Evaluation Workshop (which is part of MISRGO training efforts) have been extensive, tailoring national talents to local needs. It has attracted prospective participants from as far away as Liberia and Ethiopia, as well as throughout the United States. It will be held July 10-11, 2008. In addition to these efforts, a certificate and masters program in evaluation is being planned. For a more detailed account of evaluation efforts see the complete report or contact Dr. Fetterman.

Clearing the Air in Communities of Color 2008

Clearing the Air. This year's conference began with Bishop Miles and a talk about "A Vision of a Smokefree Society." The theme was maintained throughout the conference with Kevin Collins (CDC) emphasizing capacity building and Onjewel Smith discussing tobacco industry techniques. Success stories were shared including presentations by Elroy Brown, Larry Owens, and Carolyn Yarbrough. Dr. David Fetterman provided a status update and workshop about MISRGO's empowerment evaluation efforts. He highlighted both the Evaluation Monitoring System and the Visual Alert Systems. Other speakers included: Lloyd Franklin (law enforcement), Ruby Trotter (youth), Misty Paschall (parent), and Bob Mathis (state government). Special presentations were made by: Wilma Houston (Asian Pacific), J.R. "Pinky" Few (Rural Community), Keisha Grigsby (African American Community), and Ruben Arana (Hispanic Community), as well as Advantage Communications. Dr. Benjamin helped to open and close the conference with the AR State PRIDE Youth Performance.

Spring Evaluation Training Workshop - April 24, 2008

Spring Evaluation Training Workshop

Grantees and evaluators came together on April 24, 2008 to participate in evaluation training. Bringing both of them together ensures that they are on the same wave-length about data needs, roles, and relationships. (A copy of the slides used to guide the training day are available on this blog.)

The training was geared toward a new monitoring tool – the Visual Alert System. (This tool is described in the Visual Alert System posting below this one.)

This tool helps monitor grantee progress across programs, providing a readily available update upon demand. For example, bar charts are used to visually document how many grantees are using the Quitline.In addition, grantees and evaluators agreed that the same bar chart format could be used to document individual grantee progress. For example, a grantee would have:

1. The first bar chart would document their baseline of 4 news articles printed so far this year.
2. The second bar chart would document their aspirations – a benchmark number such as 10 new news articles by the end of the year.
3. A third bar chart would document the actual number of articles they had published that year, which might be 8.
4. A fourth chart would compare the first baseline bar chart of 4 with a final end of the year bar chart of 8 – documenting their progress throughout the year.

These bar charts could be forwarded to MISRGO and Dr. Fetterman and Ms. Delaney to document progress across grantees based on activities held in common, such as the Quitline or newspaper articles.


Dr. Johnson, Dean of the School of Education, was present to introduce new points and reinforce existing recommendations, including the need include activities directly related to tobacco prevention efforts even if the activities were not originally listed in the plan. In addition, he reminded grantees of the need to seek external funding.


The Visual Alert System provided an insight into patterns across grantees, alerting grantees and evaluators to successes that should be reinforced and continued, as well as areas meriting attention. The issue of whether we were implementing the “right” or most effective activities was discussed.

The Center for Disease Control document concerning Best Practices helped guide this part of the discussion. A summary of best practices was distributed during the workshop. CDC’s complete document is available on line.
One of the Visual Alert System findings initially suggested that few grantees were applying CDC Best Practices. However, after some discussion it was clear that grantees were in fact applying CDC Best Practices without knowing it, e.g. Quitline, coalition building, and so on. This finding and discussion highlighted the importance of sharing CDC documentation and engaging in a dialogue about what was being implemented in practice.

The Evaluation Monitoring System was also emphasized during the workshop, highlighting the importance of the group continuing to work together as a group. This system has demonstrated how much money they have saved as a unified tobacco prevention coalition - $84 million.











The workshop concluded with a few announcements including the:

1. establishment of the Arkansas Evaluation Center
2. development of an evaluation certificate program (and master’s program) at UAPB
3. upcoming Arkansas Summer Evaluation Institute Workshop – July 10th and 11th

The group responded enthusiastically to the announcements.

Dr. Fetterman pointed to the LCD projection to let them know the details associated with the Summer Evaluation Workshop.

The Summer Evaluation Workshop will include:

• Senator Tracey Steele
• CDC’s Health Education Specialist Kevin Collins
• Dr. Stuart Donaldson, Dean of the School of Behavioral and Organizational Sciences, Claremont Graduate University - Introduction to Practical Program Evaluation (Program Theory)
• Dr. Tarek Azzam, Senior Research Associate, School of Behavioral and Organizational Sciences, Claremont Graduate University - Introduction to Technology and Evaluation: Focus on the use of Geographic Information Systems and Online Surveys
• Dr. Rosalie Torres, Torres Consulting Group - Interviewing Skills for Evaluators
• Dr. Fetterman, Director of Evaluation, School of Medicine, Stanford University and Co-Director, Arkansas Evaluation Center, School of Education, UAPB
• Mr. Charles Gasper, Director of Evaluation, Missouri Foundation for Health - A Foundation Perspective about the Significance of Program Evaluation
• Dr. Nacaman Williams, Walton Foundation – A Foundation Perspective about the Significance of Program Evaluation

The Summer Evaluation Workshop Program and Schedule is listed in under Useful Links on the left column of the blog. For your convenience, the program is also available by selecting this Summer Workshop Program 2008 link.

The Spring Evaluation Workshop concluded with an announcement about the new Arkansas Group of Evaluators. AGES is an official affiliate of the American Evaluation Association.

Their first official meeting as an affiliate will be on July 10th in the evening at UAPB. It will be held during the Summer Evaluation Workshop. Ms. Linda Delaney is the chair of the State-wide evaluation group.

Visual Alert System

Visual Alert System. We have developed a system that enables us to monitor our progress quickly. We are using a simple visual tool - bar charts. The MISRGO Mid-Year and Annual Evaluation Reports can be difficult to navigate. There are 18 reports covering varying topics ranging from coalition building to promoting the use of the Quitline. It is a time consuming and laborious task to compare grantee efforts throughout the State by reading the text in each report alone. We have applied this tool to the current Mid-Year Evaluation Reports (see Visual Alert System listed on the right hand column of this blog). The bar charts enable us to quickly scan all of the reports and determine how many of us are building coalitions, using the Evaluation Monitoring System, working with churches to promote tobacco prevention activities, and so on. Grantees can look at the bar charts and decide to shift their efforts to strengthen the groups' efforts as a collective. Policymakers and legislators can also see where we are putting our emphasis to meet our goals. We will provide training on the use of this tool during our upcoming training session (April 24, 2008). This tool among others will be highlighted during our Arkansas Summer Evaluation Institute Workshop, which will be held at the University of Arkansas at Pine Bluff on July 9-10, 2008.

Evaluation Technology Workshop Sept 27, 2007

Evaluation Technology Workshop. MISRGO provided a technology training day at the University of Arkansas at Pine Bluff. The training was designed to familiarize grantees and evaluators with a variety of web-based tools that will help them with their programs and assessments of their programs.

Dr. Fetterman was invited to facilitate the session.

He provided an overview of useful tools, including online surveys, blogs, digital photography, picture file sharing, internet telephony (Skype), online documents and spreadsheets, and videoconferencing on the net.

Grantees and evaluators will be using the online spreadsheet to keep track of the Evaluation Monitoring System data. The Evaluation Monitoring system keeps track of the number of people who have stopped using tobacco as attributed to grantee efforts and translates that figure into dollars saved concerning excess medical costs for the State). This will allow MISRGO, grantees, and evaluators to access the data on demand.



The day was energizing as we interactively created online surveys and blogs and demonstrated the power of the net.


We also demonstrated the value of online calendars to announce event and schedule meetings across grantees and evaluators. Dr. Fetterman's slides are available here. In addition, Dr. Fetterman created a pre-test survey and a post-test survey for participants to help evaluate the session. The results were extremely positive, documenting gains in familiarity and knowledge about online surveys, blogging, online documents and spreadsheets, file sharing (pictures), telephony, and videoconferencing on the net.

In addition, a few emails provide some insight into the participants’ assessment of the workshop: “Thank you, Dr. David, for the session last week. I will put into practice all I have learned.” Another said: “Hi. Thank you for your patience and thoughtfulness with us!!! Thanks also for the blogs!!!” A copy of the Evaluation Technology Workshop Report is available from this blog.
The participants co-created another survey. It was the Arkansas Evaluation Center Needs Assessment survey. The results of the brief needs assessment is being used to inform the Center's curricular development .

MISRGO EVALUATION PLAN - 2007-2008

MISRGO Evaluation Plan. This document is divided into two sections. The first half highlights accomplishments to-date, including creating the Evaluation Monitoring System.

The second half of the report presents the evaluation plan for this coming year. It charts the way for us. We plan to use the report to help us navigate through often mirky and tumultuous waves. It also provides an insight into what's in store for us in the future.

For a copy of the report click here.

MISRGO Evaluation - Grantee Satisfaction with Office Services

MISRGO provides administrative and support services to tobacco prevention programs in Arkansas. Fetterman & Associates are currently facilitating an empowerment evaluation of the program, helping grantees use evaluation to accomplish their objectives (with a continuous cycle of assessment and action).

One part of the evaluation includes assessing MISRGO itself, as a supporting office or set of services. Refinements and improvements in the Office directly benefit and enhance grantee work. MISRGO staff embraced this part of the assessment and helped generate questions for the online survey because they sincerely wanted to know what grantees thought about their services. In addition, they wanted to improve the quality of their services to grantees. They understood the chain of reasoning involved, enhancing services increases the probability of improving grantee success in meeting their tobacco prevention objectives.

The report is available to the public. It was generally favorable, they valued the services they have been receiving. However, they were also critical and suggested areas meriting attention. They also recommended a list of ideas for future workshops and services.

MISRGO is already processing this information and planning to adopt many of the recommendations associated with the report, including soliciting more involvement from grantees in future conferences. This is the first step in evaluating the office, additional activities will include a site visit, interviews with staff members and grantees, and a review of relevant documentation.

4th Annual Conference Clearing the Air in Communities of Color May 23-24, 2007

4th Annual Conference" Clearing the Air in Communities of Color

The conference received letters of support from Governor Mike Beebe, Mayor Carl Redus, Chancellor Lawrence Davis, Jr., Dean Calvin Johnson, and Program Coordinator Marian Evans.

Keynote Speakers included: Senator Henry "Hank" Wilkins, Broadway Joe Booker, and Kenya Masala.

Sessions included: Tobacco Use in the Asian Pacific Community, The Oxygen Project, Tobacco Use Among African American 13 Year Olds in Jefferson County,

Cease Project, Best Practices, Advocacy and You, Fire and Burns Related to Cigarettes, Improve Service Delivery by Embracing Youth Culture, Empowerment Evaluation Principles, Tobacco Control in the Latino Community of Arkansas, and Just Say Y.E.S. on the Move for Better Choices.


Dr. Fetterman presented the Closing Luncheon Presentation.

He provided conference participants (particularly grantees and evaluators), with an update on the status of the evaluation, including the Evaluation Monitoring System, the State-wide Community Survey Results, and the new Arkansas Evaluation Center.

In addition, a special video was presented highlighting the accomplishments of each grantee. (It was both inspiring and humorous as well.)

Number Quit Smoking, MISRGO Office Evaluation & the Arkansas Evaluation Center - April 26, 2007

Evaluation Workshop in Little Rock - April 26, 2007

The April MISRGO Evaluation Workshop in Little Rock was very productive. Dr. Fetterman and Ms. Linda Delaney facilitated the workshop. The participants brought the session alive. They were completely engaged. (Click on picture above to display a brief video of our workshop.)
















There were three key topics covered: the evaluation monitoring system (focusing on the number who quit smoking or quit using tobacco); the evaluation of the MISRGO office; and the Arkansas Evaluation Center. (Click on Fetterman & Associates below to access the slide presentation during the workshop.)

Evaluation Monitoring System

The evaluation team was reminded of the importance of continuing to collect tobacco consumption (or in this case cessation) data. They agreed an ideal data collection effort would collect monthly figures. However, a realistic goals was to assist the grantees as they collect quarterly and bi-annual cumulative data. Evaluators plan on reminding grantees how important it is to continue to collect the data, even if they only record one person who stopped smoking, since that represents $150,000 in excess medical costs saved.

Dr. Christine Sheffer
University of Arkansas for Medical Sciences
Fay W. Boozman College of Public Health
Health Behavior and Health Education Department
SOSQuitline 1.866.NOW.QUIT
SOSWorks Fax 1.888.827.7057

Marian Evans, MISRGO Coordinator, stated she would follow up with Dr. Christine Sheffer, University of Arkansas for Medical Sciences, concerning the SOS data referral plan, to help us track grantee referrals to smoke cessation programs.


MISRGO Office Evaluation

This was one of the most rewarding parts of the workshop. The focus was on plans to evaluate the effectiveness of the MISRGO office, in terms of supporting and facilitating grantee and evaluator activity.

MISRGO office staff members and MISRGO/grantee evaluators worked together to generate a list of useful questions to assess ourselves. Instead of the typical defensiveness associated with endeavor, the MISRGO office staff jumped in, posing difficult questions that they wanted the grantees and evaluators to ask about their performance. They did this because they want to know what they can do better, to better serve the entire community. It was a rewarding and fruitful exercise. An online survey will be developed in the next few weeks and grantees and evaluators will be asked to provide their views concerning the MISRGO office's efforts. This is in the true spirit of empowerment evaluation and self-assessment (to improve and enhance).

Sample Questions

Arkansas Evaluation Center

Dr. Fetterman and Ms. Delaney announced and described the new Arkansas Evaluation Center (and highlighted the new blog page about the Center). The bill (now an act) passed the House and Senate and was approved by the Governor.



Dr. Fetterman and Ms. Delaney took this time to solicit input from the workshop participants concerning the overall design of the Center, possible courses, and the strengths and weaknesses of the online distance learning component (using WebCT).


The participants were generative and productive. They recommended specific courses for a certificate program, such as research design, survey design, focus groups, interviews, and statistics (light). They also recommended a course about ethics, guidelines, and standards.

Based on a straw vote, everyone said they would be interested in participating in a evaluation certificate program. The next steps will include: the development of a needs assessment survey - to determine the level of interest in the program (above and beyond our group), a review of comparable programs (to identify the best features across programs that we might incorporate in our program), and course development (including WebCT distance learning programs).

Arkansas Evaluation Center

The Minority Initiative Subrecipient Grant Office (MISRGO) and the Evaluation Team proposed the creation of an Arkansas Evaluation Center to the Arkansas Legislature on March 19, 2007.

The idea emerged from our tobacco prevention work. The need for more evaluators and more systematic evaluation training throughout the State became increasingly evident. Not only were more evaluators needed for tobacco prevention work but for many other programs as well. Thus our proposal to establish a center to train new evaluators and provide professional development training for working practitioners.

(Click on Picture to Display Full Size Poster of Arkansas Evaluation Center)

The bill describes the mission or purpose of the center. Highlights include:

• enhancing the quality of existing services and programs
• ensuring accountability for funds and services used
• “raising the bar” concerning standards and expectations
• increasing the use of data to inform program operation and decision making.

(Click on picture of Bill to display full PDF of the Bill.)

An empowerment evaluation approach and philosophy has been adopted by the Center to shape practice in communities throughout the State. Empowerment evaluation and other collaborative forms of evaluation are particularly useful in complex collaborative endeavors requiring the participation of diverse groups. It is designed to help groups accomplish their objectives using evaluation concepts and tools.

Special thanks are due to all of the MISRGO staff, grantees, and evaluators for helping to identify this need and transform this need into an action step designed to meet the need.

November 4, 2006 - American Evaluation Association (dissemination)

Dr. Fetterman and Ms. Linda Delaney helped to disseminate the Tobacco Prevention evaluation findings at the American Evaluation Association in November, highlighting the amount of money saved the State as a result of these tobacco prevention programs. They also discussed the politics of tobacco prevention evaluations.

(click on photo above to display video)

September 22, 2006 - Refresher, Focus Group, and Black Caucus

The September 22 workshop was designed to be a refresher and provide additional methodological training. Dr. Fetterman provided an overview of the project and its current status from an evaluation perspective, including the evaluation monitoring system and the community survey. Dr. Fetterman also provided focus group training for the Arkansas MISRGO evaluators. Ms. Delaney faciliated the self-assessment process of the training generating a 4th data point of self-assessment for the empowerment evaluation exercise.



Dr. Fetterman was also invited to meet with the Majority Leader, Senator Tracy Steele, and the Black Caucus during this trip.


He provided them with pertinent data, ranging from the amount of money saved as a result of these programs to the wide-spread community support enjoyed by MISRGO across the State (large constituent base). Dr. Fetterman also met with various news organizations to share the evaluation monitoring system findings and the community survey results.

Evaluation Monitoring Tool Update

We now have an interactive spreadsheet that we share on the web. It is our repository of tobacco consumption data. It provides us with an up-to-date view of: number of materials distributed, number pledge to quit, number who quit, cost saved per person, and cost saved by grantee group. We also have a running total of each of these categories including total cost save the State concerning excess medical costs. As of July 4, 2006, the totals across the State are as follows:


We also maintain a more detailed data sheet, enabling us to view the numbers by grantee (see below).

Community Surveys

We conducted a community survey throughout the State of Arkansas focusing on the views of the grantees' community members. We received 946 responses. The findings were overwhelmingly positive.

A few survey responses are provided below to provide some insight into the level of community support for tobacco prevention programs in the State.


In addition to providing information in the aggregate about community views across the State, we produced a report that provides community member viewpoints about their specific grantee or tobacco prevention program.

Clearing the Air in Communities of Color Conference

We shared the evaluation approaches we have been using to measure change with colleagues at the 3rd Annual Clearing the Air in Communities of Color conference. (See video below.)

(click on picture to display video)

We discussed our training efforts, the empowerment evaluation process, our evaluation monitoring system, and our community surveys. It is a useful summary of our efforts up to May 26, 2006. The slides of our presentation are available to the public.

January 2006

We held a local evaluator workshop in January. It focused on how to use the Evaluation Monitoring System. They will work with the local grantees to help them collect critical baseline data, concerning the number of people smoking in their communities. In addition, it will help programs document the number of people they reach and the number who quit. The system will also translate that into costs saved. [The workshop slides are available on this site.]

A brief video of the training is provided below.

Evaluation Monitoring System and Community Survey
Local Evaluator Workshop Training
(click on picture to display video)

In addition, we are working on community surveys of the grantees so that this information can be fed back into the programs to enable them to enhance their service to their communities.

We will continue to use the Evaluation Monitoring System tool, along with many others, and rate our own performance throughout the evaluation. These tools help to guide us through the journey.

September 2005 - Common Indicators and Qualitative Methods

The September workshop was extremely productive. The grantees agreed upon common indicators to measure program performance, including the size of the target population, the number of smokers, the number reached with educational messages and services, and the number pledged to quit (as well as documented individuals in smoke cessation programs). A sample is provided below (see Figure 1):

Figure 1. Common Indicators

Dr. George Blevins, MISRGO Director,

introducing the September 2005 workshop.

Grantee oriented interviews and surveys were conducted previous to the workshop to solicit grassroots data concerning how grantees collect data, the quality of the data they have, and program implementation. In addition, recommendations were made during the interviews and workshop, including how to collect sign-in and sign-up sheets as well as pledge to quit sheets. Census and At Risk web site were shared to help grantees collect valid and useful data concerning the smoking rates in their communities.

Risk Factor and Census reports shared with grantees

The workshop focused on how to use various qualitative techniques such as ethnographically-oriented observation and interviewing. The workshop also demonstrated how to construct online surveys (with real examples on the web).

Grantees completing an ethnographically oriented observation exercise
and Dr. Fetterman critiquing their work.


The slides from the workshop training are available in pdf format. The Final Report for Year One summarizing work-to-date is online and available on this site. In addition, a brief video is available concerning workshop highlights.

Brief video of September 2005 workshop
focusing on common indicators and
qualitative techniques for data collection.

(Select arrow on bottom left of the screen to display video)

During the meeting we agreed on the need to develop an Evaluation Monitoring System. (See Figure 17 below.) It is described in the September 2005 Annual Report.

We also thought it might be useful to write a Local Evaluator Training Manual. This will help us create greater consistency in practice, ranging from collecting the same type of data to analyzing it in the same format.

We continued to operate as a group or consortium. We have adopted an empowerment evaluation approach to guide our work. This approach enables us to monitor our progress at each stage in our development as a group. We "check in" each time we meet and assess our progress. In this case, we have documented the third data point or assessment of our progress to date (see graph below).