Social Justice & Evaluation Conference Presentations & Abstracts

Session Abstracts

Living-Into Social Justice Praxis: Cultivating Self-in-Context as Responsive Evaluators
Hazel Symonette, Program Development & Assessment Specialist in the University of Wisconsin-Madison Division of Student Life, Founder and Director of the Student Success Institute

Too often, well-meaning, well-intentioned services have unintended problematic impacts that result in the social costs of accepting help being too high. We increase prospects for social justice praxis and for operating at our evaluator best when we intentionally embrace a culturally- and contextually-responsive action researcher stance. Doing so involves systematic data-grounded inquiry as an ongoing, evidence-framing dialogue with SELF-in-Context as evaluator--vis a vis one’s stakeholders and the requirements of the evaluation agenda. For excellence and ethical praxis, evaluation services should be broadly diversity-grounded and equity- minded. They should be socially-responsive, socially-responsible and socially just as informed by the American Evaluation Association’s Guiding Principles and the Joint Committee on Standards for Educational Evaluation’s Program Evaluation Standards. My holistic systematic inquiry and reflective practice framework promotes empathically scanning, tracking and unpacking WHO? factors using an Integral Quadrant model: who is served by whom with whom as embedded in situational, relational, temporal and spatial/geographic contexts. This Activist Assessment & Evaluation approach is a resource for enhancing Interpersonal Validity vis a vis responsive programmatic design, development and assessment. This workshop will also include a brief update on the American Evaluation Association Task Force on Evaluator Competencies and information about feedback opportunities for revising the final draft before distribution to and review by the full AEA membership.

Meet Me at the Corner of Evaluation & Politics: Evaluators Role in Shaping Change

Laura Pinsoneault, Senior Evaluator for the Advancing a Healthier Wisconsin Endowment at the Medical College of Wisconsin, previous Director of Research for the Alliance for Children and Families, co-founder of ¡Milwaukee Evaluation!, Inc.

Evaluation is inherently political. This condition positions us, as Evaluators, with a great deal of influence and power over which voices get priority and how resources get distributed. Evaluators working closely to advance social justice and advocate for inclusion may be questioning how to stay authentic and how to “think and work politically” within our discipline. This Roundtable will encourage frank discussion from all perspectives on how to navigate our roles as evaluators both nationally and within our own daily political contexts while being true to the change each of us hopes to influence.

Introduction to Evaluation

Libby Smith, External Evaluator at the University of Wisconsin-Stout Applied Research Center, Program Director for the MS in Applied Psychology program, teaches Evaluation Ethics & Practice in the Evaluation Studies Graduate Certificate at UW-Stout

New to evaluation? Learn the basics of evaluation, key concepts, and steps. Learn what it means to include social justice into an evaluation project.

“So You Want to Meet my Family?” - A Latino Nonprofit’s Approach to Culturally Responsive Research and Evaluation through Community-Academic Partnerships 

Al Castro, Program Director of the Health Research Department at the United Community Center

When researchers approach a community to conduct research or evaluations, they are basically asking to come inside to meet “the family” (the community).

If the approach is not thoughtful and considerate of what the family believes is important and how the family communicates, the researcher may not make it past the front door. Sometimes, even if invited inside, the parties may soon realize they are not fully understanding each other, become frustrated, and the visit is over.

If the relationship goes on to attempt to work together, but find out they are not working on what is really important to both, the relationship may get rocky and dissolve, or end up with something not valuable or useful to both parties. Effective community engaged research by cademic/research institutions may involve similar characteristics as a “blended marriage” in order for a successful partnership. From the initial dating to engagement, to accepting the other partner’s children and family members and their way of doing things, then marriage, creating a home and sharing the raising of step-children, working to support the family, and aging together, the two partners are tasked with sharing, learning, and growing together while learning how to communicate and managing a dynamic relationship towards common interests. These are similar dynamics encountered in community engaged research.

In this workshop you will hear the lessons learned, both good and bad, from the perspective of a community partner organization. You will learn the value of how to best approach a community, and of collaborating with community partners in an ethnic community (in this case, a Latino community) to enhance culturally effective and acceptable study designs and engagement of the target community. You will learn of what community organizations see is important to assess, change and measure, and to see beyond the research project, and how the community needs to have a voice in the efforts. You will learn about, and be able to have informal discussion, of examples of successful or promising community engaged health research projects with academic partners. The aim is to share what we have learned at the United Community Center in hopes of providing guidance in your future evaluation or research projects in underserved ethnic communities in order to provide something of long term value for that community.

Improving Learning Measures: A Gentle Introduction to Item Response Theory

Eric Hagedorn, Founder of Hagedorn Evaluation Services, University of Texas at El Paso Associate Professor of Physics and Interim Department Chair, previous Professor at University of Milwaukee

While there are many drawbacks to multiple choice tests of learning, they are, for better or worse, well established in educational policy. Item Response Theory (IRT) is a powerful quantitative tool for evaluating the effectiveness of multiple choice response questions individually and together as a test. This presentation will provide non-quantitative evaluators a working understanding of IRT analyses, including Differential Item Functioning, which determine if items function differently for different subgroups of test takers. Actual data the presenter has analyzed will be used to make examples truly real-life.

Managing Social-isms During the Evaluation Process for Early Career Evaluators

Quiton Cotton, Case Western Reserve University, Marquette University, current full-time doctoral student at UW-Madison

Social, community, organizational, individual and cultural experiences all provide contextual information relevant for evaluation. The extent to which an evaluator understands the experiences of self and stakeholder perspectives, influences his/ her ability to conduct evaluation activities successfully. Evaluators are often responsible for managing relationships with various stakeholders with experiences that are complex, rich and even suspicious of the evaluation process. The goal of this workshop is to introduce early career evaluators to facilitation techniques that can assist in managing Social ISMs or other events that may emerge during the evaluation process. Learners will be exposed to the term positionality, methods for enhancing evaluation meetings and managing the evaluation process.

What do New and Emerging Evaluators Want? Mentorship Needs and Aspirations of Early-Career Evaluators

Asthma Ali, Evaluation and Applied Research Professional at the American Society for Clinical Pathology, President of the Chicagoland Evaluation Association [CEA], currently getting Ph.D. in Urban Planning and Policy

Within the last few years, mentoring has emerged as a method for ensuring diversity and growth in the evaluation profession. At the national-level the American Evaluation Association has sponsored several programs which incorporate mentorship of diverse early career evaluators. In this session, we present the results of a study of early career evaluators from AEA’s GEDI program focusing on their mentorship needs. Using a Q-sort methodology, researchers uncovered 3 distinct groups of mentorship needs among emerging-to early-career evaluators. Participants will utilize the three-group mentorship model to participate in a group activity on mentorship needs and designs.

Building Social Justice into your Results Based Accountability Practice

Marian Amao, Senior Associate at The Annie E. Casey Foundation

  Randy Nelson, Program Coordinator for the Criminal Justice Administration Graduate Program at Bethune- Cookman University

This session will be led by Marian Amoa of the Annie E. Casey Foundation and
Dr. Randy Nelson from Bethune-Cookman University.The session will apply the Results-Based Accountability framework to the field of social justice. In doing so, the presenters will (1) review fundamental RBA concepts; (2) define and discuss the term “social justice” using an RBA frame; and (3) in an interactive format, engage session participants in applying RBA to a real-world example of racial disparities.

Pensar sobre la Evaluación– En Español

Mariana Castro, Director of Standards for WIDA at the University of Wisconsin-Madison, University of Wisconsin, Madison and Whitewater, and at Edgewood College

Antía Gonzalez Ben, Project Assistant at WIDA, University of Wisconsin-Madison, currently working on Ph.D.

This session is for Centro Hispano guests and invited Spanish-speakers to reflect on what they heard and experienced throughout the day. 

Presenter Bios