Prepared for
The Decision-Making Leaders of the
The North Dakota Guardian ad Litem Project
Prepared by
Carla Hess, Schurkey Swanke, & Annalisa Batson
HB Consultation and Evaluation Associates
Grand Forks, ND
April 30, 2007
TABLE OF CONTENTS
INTRODUCTION 1--2
The North Dakota Guardian ad Litem Project 1Purpose of this Evaluation 2
METHODS 3--7
Evaluation Design 3Participants 3--4
Instruments 5
Written Questionnaires 5
Telephone Interview Protocol 5
Procedures 6
Data Collection 6
Data and Data Analyses 7
FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS 8--55
Value of the NDGAL Project: 8Value in Various Deprivation/Abuse and Neglect Cases 8
Value in Cases Involving Specific Issues 10
Value of GALs' Input to Court Decisions 13
Value of GALs' Knowledge, Information, and Service 17
Summary and Conclusion 20
Effectiveness of the NDGAL Project: 21
Need for Documentation of Project Contexts 21
Mission, Goals, and Related Contexts 21
Policies and Guidelines 22
Roles and Expectations for GALs 23
Need to Identify the Project Decision-Making Leadership 26
Personnel Resources 26
Project Directorship 27
Number of GALs 28
Duration of GAL Appointments 29
Fiscal Resources 30
Recordkeeping 31
Effectiveness of the GALs 32
GALs' Understandings and Work Performance 32
GALs' Contacting and Interviewing Various Parties 37
GALs' Case-Related Activities 40
Training of the GALs 47
Strengths and Weaknesses of Training 48
Training on Court and Legal Systems 49
Training on Types of Cases and Issues 49
Other Topics for Training 49
Summary and Conclusion 50
Satisfaction with the NDGAL Project and the GALs 51
Summary and Conclusion 55
REFERENCES 56
APPENDIX A A Survey for Judges and Judicial Referees 57APPENDIX B A Survey for Guardians 61
INTRODUCTION
The North Dakota Guardian ad Litem Project
An early form of the North Dakota Guardian ad Litem (NDGAL) Project came into existence in 1986. However, during the initial 14 years, organizer Tara Muhlhauser, now Deputy Director of the Children and Family Services Division and Program Administrator for Child Protective Services, North Dakota Department of Human Services, worked informally with the North Dakota Supreme Court in envisioning a system for involving guardians ad litem (GALs) in child deprivation cases. By 2000 the groundwork had been laid for securing core funding to begin the NDGAL Project formally. Ms. Muhlhauser served as the first project director until January 2003. During her tenure the NDGAL infrastructure was formed based on pilot projects in Fargo and Grand Forks; guardians ad litem were encoded in the North Dakota Century Code (Chapter 28-03); and the first staff was hired. Subsequently, the project expanded throughout the state.
Presently, under the direction of Mr. Brad Swenson with coordination of the GALs by Mr. Lanny Serrano, the Project lists 45 GALs across the state with representation in all Juvenile Court regions. The project is now funded with Federal Court Improvement grant money administered through the North Dakota Court Administrators Office.(1)
North Dakota GALs are paid independent contractors working at an hourly rate. GALs have not been trained in law. They represent many professions. However, GALs do receive an initial 18-hour training and ongoing training is available.
The last initial training for new GALs occurred in Fall 2006. According to the "Training Agenda," the topics covered included: 1) history of the project; 2) duties of a GAL; 3) role/responsibilities of a GAL; 4) professionalism; 5) limits; 6) confidentiality; 7) skills, values, and attitudes; 8) child protection; 9) foster care; 10) introduction to the Juvenile Court; 11) the Adoption and Safe Families Act (ASFA); 12) the Indian Child Welfare Act (ICWA); 12) domestic violence; and 13) substance abuse. Each GAL also received additional printed resources in the form of outlines, articles, and papers in a large ring-bound notebook. These materials addressed such topics as 1) "Guidelines for Being an Effective Witness"; 2) "Principles & Concepts that Guide CASA/GAL Volunteer Work"; and 3) "Confidentiality"; as well as issues involved in specific types of cases such as cultural competence and dealing with differences including lesbian, gay, bisexual, and transgender youth in foster care.
Ongoing GAL training takes the form of monthly regional meetings. At these sessions the GALs receive updates, discuss cases, and hear presentations from community representatives.
Historically, there have been no general conferences for this program, although some GALs have attended what they perceived to be GAL-relevant conferences at their own expense. The first statewide meeting for GALs is currently being planned for July 2007.
The NDGAL Project has undergone previous evaluation and study [a 2000 evaluation by the Child Welfare Research Bureau at the University of North Dakota (UND) and a 2006 graduate student thesis by Dana L. Doan]. However, the UND Department of Social Work, which headquarters and administers the NDGAL Project, proposed additional evaluation in the 2006-2007 Statewide Guardian ad Litem Project Proposal for the North Dakota Supreme Court. As stated in that proposal, "After the dramatic growth of the program, it is felt it would be appropriate to undergo the process of evaluation again" (p. 5). The proposal also reports that
The focus of the evaluation will be on the stakeholders' perceptions of the services provided by the independently contracted guardians ad litem. This will include both qualitative and quantitative data to assess general perceptions. A final document will be provided to appropriate stakeholders with suggestions for improvement noted (p. 8).
Purpose of this Evaluation
In planning meetings with evaluation team members, GAL Project administrators defined the purpose of this evaluation:
Based on the analysis of qualitative and quantitative stakeholders' perception data, determine the 1) value and 2) effectiveness of the GAL Project and 3) the levels of stakeholder satisfaction with the project.
The core evaluation question posed informally by administrative stakeholders was "How are we doing, and what can we do better?"
METHODS
Evaluation Design
To guide this study, the evaluation team selected a standard formative, decision-oriented evaluation design, the CIPP Model (Stufflebeam & Shinkfield,1985). This design focuses on gathering information for use by decision makers seeking to improve their established program. The components of interest in the CIPP Model are these:
Context evaluation which serves planning decisions addressing issues such as need for the program; mission and goals of the program; and policies, guidelines, and strategic plans of the program.
Input evaluation which serves structuring decisions addressing issues such as social, fiscal, and institutional support for the program; infrastructure; personnel; and other resources.
Process evaluation which serves implementing decisions addressing issues such as program procedures, program activities, and personnel training.
Product evaluation which serves recycling decisions addressing issues such as program attainments relative to need, mission, goals, and plans; program sustainability; program continuance with expansion or contraction; and program discontinuance with dissolution decisions.
Although the CIPP components will be referred to throughout the remaining sections of this report, the findings and recommendations are organized below according to the purposes of this evaluation: to determine value, effectiveness, and satisfaction.
Participants
This evaluation was based on survey research. Both quantitative and qualitative perception data were obtained via printed questionnaires and telephone interviews. Six groups of stakeholders participated in this evaluation:
6) Fifteen judges and judicial referees--often referred to as "judges" throughout the remainder of this report;
7) Eleven state's and defense attorneys--grouped together as "attorneys" throughout this report;
8) Sixty-five child welfare workers consisting primarily of county social workers but also including two foster care workers and one Social Services supervisor--collectively referred to hereafter as "child welfare workers" (CWWs);
9) Twenty-seven guardians ad litem--referred to as GALs;
10) Three current or former GAL Project administrators; and 6) two Supreme Court administrators--referred to hereafter collectively as "Project administrators."
Printed questionnaires were sent to and returned by the numbers and percentages of stakeholders in the four participant groups identified in Table 1.
Table 1. Numbers Of Questionnaires Sent and Returned with the Percentage Return Rate per Respondent Group.
| Group | Questionnaires
Sent (#) |
Questionnaires
Returned (#) |
Return
Rate (%) |
| CWWs | 167 | 65 | 38.9 |
| GALs | 45 | 27 | 60.0 |
| Judges | 30 | 15 | 50.0 |
| Attorneys | 30 | 11 | 36.7 |
| Totals | 272 | 118 | 43.4 |
A few returned questionnaires were missing responses to some items. The data provided by these respondents were included. Six individuals returned questionnaires devoid of data. These individuals were not counted as respondents in Table 1.
One type of questionnaire response option was Not applicable (N/A). The numbers of these responses were not reported; N/A responses were treated as missing data. Missing data and N/A responses explain the unequal numbers of respondents sometimes reported for successive items in the tables and figures.
The evaluators conducted telephone interviews with 32 stakeholders representing the six respondent groups including: three judges, four attorneys, ten CWWs; ten GALs; three project administrators, and two court administrators.
Both the questionnaire respondents and telephone interviewees were well-distributed across the state. Based on their demographic information, these participants represented jurisdictions ranging from 1) relatively large to small child deprivation caseloads, 2) large to small populations, and 3) large to small geographical areas. The tenure of the participants within their various professions also ranged from short durations of one to five years to long durations of more than 25 years. Overall, the evaluators were satisfied that the participants in this study were appropriately representative of their respective professional peers.
Instruments
Two types of instruments were used in this evaluation. Written questionnaires were sent to participants via the postal service. A telephone protocol was administered orally by the evaluators.
Written Questionnaires
The evaluators developed a core questionnaire adapted from an instrument pilot-tested and administered by the National Court Appointed Special Advocate (CASA) Association in a 2003 study of "the impact of Court Appointed Special Advocates/ Guardians ad litem (CASA/GAL) volunteers and program activities on judicial decision-making, court processes and case outcomes" (http://www.casanet.org/download/casa-surveys/CS-survey-final-report-09-03.pdf accessed on February 20, 2007). For the present North Dakota study, the evaluators subsequently adapted their core questionnaire to create four forms that were each appropriate in content and wording for one of the four questionnaire respondent groups. The questionnaires addressed the following topics through a series of closed-ended and open-ended questions:
Demographic factors of respondents regarding their length of service in Juvenile or Family Court deprivation/abuse and neglect cases and the numbers of their recent and current cases;
The usefulness of the NDGAL Project;
Roles of GALs in supporting judicial decision-making and court processes;
Effectiveness of the NDGAL Project and GALs;
Satisfaction with the NDGAL Project and GALs; and
Suggestions for improving GALs' service to children and court decision- making.
The questionnaire sent to judges and judicial referees is provided in Appendix A. Its content is similar to that of the questionnaires sent to attorneys and CWWs. The questionnaire for the GALs was worded for the GALs' perspective and included somewhat different demographic items (see Appendix B).
Telephone Interview Protocol
The interview protocol consisted, in part, of the following four guide questions:
How do GALs contribute to the welfare of the children they serve?
What are the greatest strengths of the GALs?
What could be changed to improve the NDGAL Project?
What else do you want to tell us about the NDGAL Project or GALs?
The evaluators also asked additional probe questions formulated to elicit either clarifications or expansions on the interviewee's feedback to open-ended questionnaire items or to their interview responses. Although the interview protocols were individualized, they elicited information directly relevant to the purpose of this evaluation.
Procedures
Data Collection
Prior to the data collection phase of this evaluation, NDGAL Project Director Brad Swenson contacted key representatives of the various respondent groups to inform them of the upcoming evaluation and to request maximum participation by their groups. Subsequently, Mr. Swenson provided the evaluators with contact information leading to the identification of names and mailing addresses of potential survey respondents. The survey mailings included a cover letter explaining the project, a copy of the questionnaire, and a postage-paid return envelope.
Procedurally, only individuals who agreed to be contacted were interviewed by telephone. These respondents identified themselves and provided contact information at the end of their written questionnaires. Not all individuals who agreed to be contacted were actually interviewed. The evaluators conducted interviews in the approximate order of the return of questionnaires. Interviewing continued until no new themes were emerging in the participant feedback, and the categories of data supporting those themes were "saturated, that is so well defined that there was no point in adding further exemplars" (Lincoln & Guba, 1985, pp. 343-344). Saturation was reached when "additional information was consistently redundant" (Bogdan & Biklen, 2003, p. 62).
Data and Data Analyses
The quantitative data in this evaluation consisted of the respondents' ratings of the variables included in the closed-ended questionnaire items. The ratings were elicited by the following five-point scales anchored at points 1,3, and 5:
1 = Not very useful, 3 = Somewhat useful, 5 = Very useful;
1 = Not very much, 3 = Somewhat, 5 = A great deal;
1 = Almost never, 3 = Sometimes, 5 = Almost always;
1 = Strongly disagree, 3 = Neutral, 5 = Strongly agree;
1 = Not effective, 3 = Somewhat effective, 5 = Very effective; and
1 = Not satisfied, 3 = Somewhat satisfied, 5 = Very satisfied.
The quantitative data were analyzed for percentage of respondents selecting each numerical rating 1-5 and for mean ratings by all respondents combined for each item. In order to provide easily read and interpreted findings, only numbers and percentages of respondents and mean ratings by various respondent groups are provided in the results below. Additional statistics (e.g., standard deviations, ranges, confidence intervals, and tests of significant differences with associated degrees of freedom) are available upon request to an evaluation team member.
Qualitative data were obtained from two sources: telephone interviews and open-ended questionnaire items. There was a high level of overlap in the respondent feedback obtained by these two procedures. Therefore, the qualitative data from the two sources were combined for analysis and interpretation, and the evaluators conducted content analysis by employing a constant-comparative method (Lincoln & Guba, 1985, pp. 336-351).
Qualitative data are provided in this report as bulleted statements. In some cases to improve interpretation and readability, the evaluators have inserted bracketed phrases into quotations. In order to provide a document of reasonable length, the evaluators did not include all of the raw qualitative data in this report. The participant quotes chosen for inclusion met one of the following selection criteria: 1) represented frequently-provided stakeholder input; 2) provided a completely novel point of view; or 3) offered insights to inform administrative decision-making, that is, advanced a potential recommendation or solution.
FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS
Value of the NDGAL Project
One way of determining the value of a program is to establish its usefulness in actually meeting the needs it was designed to address. Hence, value is equated with level of needs fulfillment. In CASA's 2003 national study of the impact of court-appointed advocates and GALs, that association identified both stakeholder-related and court-related needs that also apply appropriately to the NDGAL Project. Therefore, the current evaluation included stakeholder appraisal of the usefulness of the NDGAL Project and the GALs themselves relative to needs associated with 1) different types and issues of deprivation/abuse and neglect cases and 2) various court processes and judicial decisions.
Value in Various Deprivation/Abuse and Neglect Cases
Using a five-point scale ranging from 1 = Not very useful through 3 = Somewhat useful to 5 = Very useful, the four questionnaire respondent groups rated the extent to which the NDGAL Project is useful for cases involving sexual abuse, physical abuse, or neglect. As shown in Table 2, approximately 90% of the respondents rated the GAL Project as being from Somewhat useful to Very useful (ratings = 3-5) with only one or two individuals of the approximate 100 respondents to these three items perceiving that the Project is Not very useful (rating = 1). About 25% of the respondents rated the NDGAL Project as Somewhat useful (rating = 3).
Table 3 below provides the mean ratings of the four stakeholder groups for the data shown in Table 2. The means ranged from 3.4-3.6 for the Child Welfare Workers (CWWs) to 4.3-4.4 for the GALs. On the average, all four groups of stakeholders perceived the project to be from somewhat to very useful for child deprivation cases.
Figure 1 displays the mean data from Table 3 in graphic form for ease of visual comparison of the stakeholder groups' evaluation of NDGAL usefulness in three kinds of cases. That figure shows approximately a one-scale point differential (4.4 vs. 3.5) between the GALs' and the CWWs' mean ratings with judges' and attorneys' ratings falling about midway between.
Table 2. Number and Percentage of Respondents Indicating Levels of Usefulness of the GAL Project for Child Abuse or Neglect Cases (1 = Not very useful; 3 = Somewhat useful; 5 = Very useful).
| To what extent do you consider the NDGAL Project useful for child abuse or | 1 | 2 | 3 | 4 | 5 |
| Sexual abuse |
| # of Respondents (N = 96) | 1 | 10 | 27 | 32 | 26 |
| % of Respondents | 1.0 | 10.4 | 28.1 | 33.3 | 27 |
| Physical abuse |
| # of Respondents (N = 104) | 2 | 9 | 24 | 39 | 30 |
| % of Respondents | 1.9 | 8.7 | 23.1 | 37.5 | 28 |
| Neglect |
| # of Respondents (N= 105) | 1 | 10 | 24 | 35 | 35 |
| % of Respondents | 1.0 | 9.5 | 22.9 | 33.3 | 33.3 |
Table 3. Mean Responses of the Four Participant Groups Indicating Levels of Usefulness of the GAL Project for Child Abuse or Neglect Cases (1 = Not very useful; 3 = Somewhat useful; 5 = Very useful).
| To what extent do you consider the NDGAL Project useful for child abuse or | Judges | Attorneys | CWWs | GALs |
| Sexual abuse | 3.8 | 3.8 | 3.4 | 4.3 |
| Physical abuse | 3.9 | 3.8 | 3.6 | 4.4 |
| Neglect | 4.0 | 4.0 | 3.6 | 4.4 |
Overall, the data in Tables 2 and 3 and Figure 1 support the following finding:
Finding #1
Each of the four groups of stakeholders perceived the NDGAL Project to be of value for child deprivation cases involving abuse and neglect with the level of perceived value being greatest for GALs and least for CWWs.
Figure 1. Mean Responses of the Four Participant Groups Indicating Levels of Usefulness of the GAL Project for Child Abuse or Neglect Cases (1 = Not very useful; 3 = Somewhat useful; 5 = Very useful).
Value in Cases Involving Specific Issues
Using the same 5-point scale of usefulness, the respondents rated the value of the NDGAL Project relative to cases involving 1) conflicting case information/highly adversarial parties; 2) the status of a case in relation to ASFA; 3) issues related to reunification plans (e.g., disruption of plan, extra monitoring needed for reunification); and 4) concerns about the implementation of services. Approximately 80-90% of 102-105 respondents rated the NDGAL Project as being Somewhat useful to Very useful for cases involving these four issues. Tables 4 and 5 and Figure 2 show that the variable receiving the lowest (Not very usefuI = 1 and 2) ratings by 22% of the respondents was Project usefulness to the status of a case in relation to ASFA. About 15-16% of the respondents rated Project usefulness as a 1 or 2 for issues related to reunification plans and concerns about the implementation of services.
Table 4. Number and Percentage of Respondents Indicating Levels of Usefulness of
the GAL Project for Cases Involving Specific Issues (1 = Not very useful; 3 = Somewhat useful; 5 = Very useful).
| To what extent do you consider the NDGAL Project useful to a case involving: | 1 | 2 | 3 | 4 | 5 |
| Conflicting case information/ highly adversarial parties |
| # of Respondents (N = 105) | 3 | 8 | 23 | 35 | 36 |
| % of Respondents | 2.9 | 7.6 | 21.9 | 33.3 | 34.3 |
| The status of a case in relation to ASFA |
| # of Respondents (N = 102) | 5 | 17 | 36 | 24 | 20 |
| % of Respondents | 4.9 | 16.7 | 35.3 | 23.5 | 19.6 |
| Issues related to reunification plans (e.g., disruption of plan, extra |
| # of Respondents (N = 104) | 5 | 11 | 23 | 34 | 31 |
| % of Respondents | 4.8 | 10.6 | 22.1 | 32.7 | 29.8 |
| Concerns about the implementation of services |
| # of Respondents (N = 104) | 2 | 15 | 25 | 48 | 14 |
| % of Respondents | 1.9 | 14.4 | 24.0 | 46.2 | 13.5 |
Table 5. Mean Responses of the Four Participant Groups Indicating Levels of Usefulness of the GAL Project for Cases Involving Specific Issues (1 = Not very useful; 3 = Somewhat useful; 5 = Very useful).
| To what extent do you consider the NDGAL Project useful to a case involving: | Judges | Attorneys | CWWs | GALs |
| Conflicting case information/ highly adversarial parties | 4.3 | 3.6 | 3.6 | 4.3 |
| The status of a case in relation to ASFA | 3.6 | 3.3 | 3.1 | 3.8 |
| Issues related to reunification | 4.2 | 3.5 | 3.4 | 4.3 |
| Concerns about the implementation of services | 4.4 | 3.3 | 3.2 | 3.9 |
Figure 2. Mean Responses of the Four Participant Groups Indicating Levels of Usefulness of the GAL Project for Cases Involving Specific Issues (1 = Not very useful; 3 = Somewhat useful; 5 = Very useful).
For cases with these specific issues, Table 5 and Figure 2 show higher mean ratings of usefulness by judges/judicial referees and GALs than by attorneys and CWWs. However, overall, the quantitative data in Tables 4 and 5 and Figure 2 support the following finding:
Finding #2
Each of the four groups of stakeholders perceived the NDGAL project to be of value in cases involving: 1) conflicting case information/highly adversarial parties; 2) the status of a case in relation to ASFA; 3) issues related to reunification plans; and 4) concerns about the implementation of services. Judges/judicial referees and GALs provided somewhat higher ratings of usefulness than those offered by attorneys and CWWs
There was very little qualitative information addressing the usefulness of the Gal Project in cases involving these four issues. However, a CWW and a GAL provided their perceptions regarding cases involving adversarial parties and reunification plans. Their feedback is supportive of finding #2.
Parents often see the GAL as a "neutral" in an adversarial system, so they are less defensive and more open to discussion regarding case plans, role of a social worker, court procedure, etc.
The county is not ordered to do follow-up services, but being a GAL, I have been able to bring cases back to court and prevent the kids from being returned to the parents in dangerous situations.
Value of GALs' Input to Court Decisions
Using a five-point scale ranging from 1 = Not very much through 3 = Somewhat to 5 = A great deal, 97-104 respondents rated the extent to which input from GALs informs court processes and judicial decision-making in cases that involve the seven placement, service provision, visitation, and safety issues identified in Table 6. Approximately 75-95% of the respondents rated the extent of the impact of GALs' input as being Somewhat to A great deal for cases involving these seven issues. From Table 6 it appears that the issues on which GALs have relatively less impact (ratings = 1 or 2 by greater percentages of the respondents) include: location of placement rated low by 26%, frequency of visitation by family of origin (21%), safety of children/youth while in placement and after court dismissal (19%), restrictiveness of placement (15%), and service provision (15%).
It is noteworthy that the low ratings by 15% of the respondents evaluating the value of GAL input in informing the court on service provision is consistent with the finding in Table 5 of 15% of the respondents offering the same ratings for the usefulness of the NDGAL Project relative to concerns about the implementation of services.
Table 7 and Figure 3 reveal comparatively higher mean judges' ratings (Means = 3.8 to 4.2) of the value of GALs' input on these issues relative to the ratings of the other three groups (except for input on service provision).
The judge/judicial referee questionnaire included an item not asked of the other stakeholders; "How often do GAL recommendations become incorporated into the hearing's court order?" Using a scale ranging from 1 = Almost never; through 3 = Sometimes to 5 = Almost always to answer the question, the 15 respondents' generated a mean rating of 4.0. This finding again shows that the judges/judicial referees value the GAL Project and the GAL input for contributions that inform and impact court decisions.
Table 6. Number and Percentage of Respondents Indicating the Extent to Which Input from GALs Informs Court Decisions on Issues of Placement, Service Provision, Visitation, And Safety (1 = Not very much; 3 = Somewhat; 5 = A great deal).
| To what extent does input from GALs on the following issues inform Court | 1 | 2 | 3 | 4 | 5 |
| Placement stability and permanence |
| # of Respondents (N = 104) | 2 | 4 | 25 | 42 | 31 |
| % of Respondents | 1.9 | 3.8 | 24.0 | 40.4 | 29.8 |
| Concurrent placement with other minor siblings |
| # of Respondents (N = 100) | 1 | 8 | 34 | 38 | 19 |
| % of Respondents | 1 | 8.0 | 34.0 | 38.0 | 19.0 |
| Restrictiveness of placement |
| # of Respondents (N = 98) | 4 | 11 | 39 | 30 | 14 |
| % of Respondents | 4.1 | 11.2 | 39.8 | 30.6 | 14.3 |
| Location of placement |
| # of Respondents (N = 97) | 8 | 17 | 35 | 26 | 11 |
| % of Respondents | 8.2 | 17.5 | 36.1 | 26.8 | 11.3 |
| Service provision (e.g., physical health, |
| # of Respondents (N = 102) | 6 | 9 | 30 | 40 | 17 |
| % of Respondents | 5.9 | 8.8 | 29.4 | 39.2 | 16.7 |
| Frequency of visitation by family of origin |
| # of Respondents (N = 97) | 7 | 13 | 27 | 39 | 11 |
| % of Respondents | 7.2 | 13.4 | 27.8 | 40.2 | 11.3 |
| Safety of children/youth while in placement and after Court dismissal |
| # of Respondents (N = 102) | 7 | 12 | 30 | 36 | 17 |
| % of Respondents | 6.9 | 11.8 | 29.4 | 35.3 | 16.7 |
The qualitative data provide insight into the reasons for the participant ratings on these items evaluating the value of GALs' input to court decisions. For example, one attorney stated:
Their neutral investigation and recommendations give perspective and affirmation (at times) to petitioners and respondents.
Table 7. Mean Responses of the Four Participant Groups Indicating the Extent to Which Input from GALs Informs Court Decisions on Issues of Placement, Service Provision, Visitation, and Safety (1 = Not very much; 3 = Somewhat; 5 = A great deal).
| To what extent does input from GALs on the following issues inform Court | Judges | Attorneys | CWWs | GALs |
| Placement stability and permanence | 4.2 | 3.7 | 3.9 | 3.9 |
| Concurrent placement with other minor siblings | 4.1 | 3.6 | 3.5 | 3.8 |
| Restrictiveness of placement | 4.0 | 3.4 | 3.2 | 3.6 |
| Location of placement | 4.0 | 3.3 | 2.9 | 3.3 |
| Service provision (e.g., physical health, | 4.0 | 4.0 | 3.2 | 3.9 |
| Frequency of visitation by family of origin | 3.8 | 3.7 | 3.1 | 3.6 |
| Safety of children/youth while in placement and after Court | 4.0 | 3.7 | 3.2 | 3.7 |
Typical GAL comments supporting their positive impact on court decisions included:
The Court listens to our recommendations.
We provide information with regard to services that are available to the family that the court may not be aware of resulting in the possibility of services being court-ordered in the disposition.
GALs often can bring to the hearing concerns that have been expressed outside of court.
[GALs assure] that the Century Code is followed, especially if the attorney or judge is unfamiliar with Juvenile Law.
Checking to see if petitions are filed on time is part of it, too.
However, the following contrasting GAL statements illustrate the diversity of perceptions across the GALs:
I don't feel I ever have "a great deal" of influence on court decisions. Sometimes I feel neither the courts nor the social workers hold the GALs in very high esteem, some less than others, of course.
Sometimes we contribute minimally; Social Services and the court have made their decisions before we are able to provide recommendations. In other cases, we have a major impact.
The court listens to the GAL, but it is not always carried out by Social Services.
Figure 3. Mean Responses of the Four Participant Groups Indicating the Extent to Which Input from GALs Informs Court Decisions on Issues of Placement, Service Provision, Visitation, And Safety (1 = Not very much; 3 = Somewhat; 5 = A great deal).
Finding #2
Each of the four groups of stakeholders perceived the NDGAL project to be of value in cases involving: 1) conflicting case information/highly adversarial parties; 2) the status of a case in relation to ASFA; 3) issues related to reunification plans; and 4) concerns about the implementation of services. Judges/judicial referees and GALs provided somewhat higher ratings of usefulness than those offered by attorneys and CWWs
Finding #4
According to 20% or more of the respondents, the GALs have relatively less impact on decisions about 1) location of placement, 2) frequency of visitation by family of origin, and 3) safety of children/youth while in placement and after court dismissal.
Recommendation #1
That NDGAL Project decision makers explore this finding further to determine 1) the reasons for it, 2) the practical importance of it, and 3) any course of action to be taken.
Value of GALs' Knowledge, Information, and Service
Using a five-point scale ranging from 1 = Strongly disagree through 3 = Neutral to 5 = Strongly agree, 115 respondents rated their levels of agreement with positive statements about GALs' knowledge, information, and service as shown in Tables 8 and 9 and Figure 4. Table 8 shows that about 70-85% of the respondents agreed or strongly agreed with the statements with an additional 12-25% indicating their neutrality on these items. There were only three responses of Strongly disagree among the 345 ratings. To the more global third item, 85% of the respondents indicated their agreement (ratings = 4 or 5) that children and families are better served because of GALs' involvement.
Table 8. Number and Percentage of Respondents Indicating Levels of Agreement with Selected Statements (1 = Strongly disagree; 3 = Neutral; 5 = Strongly agree).
| To what degree do you agree or disagree with the following statements: | 1 | 2 | 3 | 4 | 5 |
| The personal knowledge that GALs have about the children/ youth in their |
| # of Respondents (N= 115) | 2 | 5 | 27 | 46 | 35 |
| % of Respondents | 1.7 | 4.3 | 23.5 | 40.0 | 30.4 |
| The type and quality of information that GALs provide me is beneficial to |
| # of Respondents (N = 115) | 1 | 8 | 23 | 47 | 36 |
| % of Respondents | .9 | 7.0 | 20.0 | 40.9 | 31.3 |
| Children and families are better served because of GAL's involvement. |
| # of Respondents (N = 115) | 0 | 3 | 14 | 50 | 48 |
| % of Respondents | 0 | 2.6 | 12.2 | 43.5 | 41.7 |
Table 9. Mean Responses of the Four Participant Groups Indicating Levels of Agreement with Selected Statements (1 = Strongly disagree; 3 = Neutral; 5 = Strongly agree).
| To what degree do you agree or disagree with the following statements: | Judges | Attorneys | CWWs | GALs |
| The personal knowledge that GALs have about the children/ | 4.3 | 3.6 | 3.6 | 4.3 |
| The type and quality of information that GALs provide | 4.3 | 3.7 | 3.6 | 4.7 |
| Children and families are better served because of GALs | 4.7 | 4.1 | 4.0 | 4.6 |
Figure 4. Mean Responses of the Four Participant Groups Indicating Levels of Agreement with Selected Statements (1 = Strongly disagree; 3 = Neutral; 5 = Strongly agree).
Table 9 and Figure 4 show that the judges'/judicial referees' and GALs' mean ratings of these three types of GAL contributions were consistently higher than the ratings of the attorneys and CWWs.
In summary, about 70% of the respondents agreed or strongly agreed that the personal knowledge that GALs have about the children and youth and quality of information that GALs provide are beneficial to court-related decision-making. Further, 85% of participants indicated that children and families are better served because of GALs' involvement. Judges/judicial referees and GALs tended to be more in agreement with these statements than were attorneys and CWWs.
Judges/judicial referees and attorneys addressed the value of the GALs' knowledge, information, and service in such comments as these:
GALs have provided information that would otherwise probably not have been available.
Much depends on the individual GAL; some are clearly thorough and involved, while others have a more perfunctory view of their role. Independent investigation is very helpful.
Basically, a good GAL is indispensable in many ways; a poor one is an impediment.
The GALs' perceptions of the value of their personal knowledge about children/youth and about the type and quality of information that they provide was embodied in such comments as:
GALs provide extensive history of a family, such as if they have been in the system and through the court process before.
Our written reports for pre-trial are often used by attorneys for their cases.
GALs find out information not obtained by the county like finding children that could not be found by the county and following a child not having follow-up through County Social Services.
CWWs expressed widely differing perceptions of the GALs' contributions of knowledge and information:
I feel they do a great job. However, our court doesn't let the GALs talk much in court.
The GALs I've worked with gave clear, concise presentations on why a child should not return home. They had facts to support their recommendations.
GALs that are more educated could have more weight in court.
A GAL Project administrator summarized well the perceptions that children and families are better served because of the involvement of GALs:
I think it's good for the child or youth himself. GALs provide emotional support, and then the child knows somebody is there for him--assigned to him. It gives them the sense someone's looking out for them. It brings stability to the child emotionally. It's good for the rest of the family, too. It provides assurance that the right things are being done.
In summary, based on the data reported in this subsection on the value of GALs' knowledge, information, and service to the decision-making of the four stakeholder groups, the evaluators found that:
Finding #5
There was a high level of stakeholder agreement that the personal knowledge that GALs have about children and youth and the quality of information that GALs provide are beneficial to court-related decision-making.
Finding #6
There was a high level of stakeholder agreement that children and families are better served because of GALs' involvement.
Finding #7
The levels of agreement of the judges'/judicial referees' and GALs' relative to the three types of GAL contributions in Findings #5 & #6 were consistently higher than the levels indicated by the attorneys and CWWs.
Summary and Conclusion
In this evaluation, the value of the NDGAL Project was equated with its usefulness in actually meeting the needs it was designed to address. CASA has identified both stakeholder-related and court-related needs associated with 1) different types and issues of deprivation/abuse and neglect cases and 2) various court processes and judicial decisions. In this study stakeholders evaluated the usefulness of the NDGAL Project and the GALs themselves relative to these stakeholder-related and court-related needs. Based on the evidence reported in this section on the value of the NDGAL Project, the evaluation team has concluded: Conclusion #1
As measured against a criterion of needs fulfillment, the NDGAL Project and the GALs are making valuable contributions to meeting the needs associated with 1) different types and issues of deprivation/abuse and neglect cases and 2) various court processes and judicial decisions.
Effectiveness of the NDGAL Project
In accordance with the design selected for this evaluation, three components of the CIPP Model were applied to the following study of the effectiveness of the NDGAL Project: context evaluation, input (resource) evaluation, and process valuation (see p. 3 for definitions of these components).
One group of criteria commonly used in evaluating the context of a program is the extent to which the program 1) is meeting an identified need; 2) is pursuing a stated mission; and 3) is achieving needs-based, mission-driven goals. Further, the effectiveness of many programs is determined relative to the accomplishment of measurable objectives and the level of success in implementing objectives-oriented strategic and/or annual action plans. Finally, context evaluation appraises program implementation relative to policies and rules or guidelines that were formulated, adopted, and subsequently updated, as deemed appropriate, by the program's decision makers.
Need for Documentation of Project Contexts
Mission, Goals, and Related Contexts
Apparently, the NDGAL Project does not have a written statement of its mission and goals. Further, the Project does not operate on the basis of written objectives, a strategic plan (except within the 1996 proposal seeking funding), or policies and guidelines. There are apparently no procedures in place for ongoing personnel evaluation of the Project director, Project coordinator, or the GALs. These types of contextual infrastructure have two crucial functions: 1) they provide benchmarks against which to evaluate a project to generate evidence-based support for funding, publicity, advertising, and other purposes; and 2) they serve as the basis for planning decisions, structuring decisions, implementing decisions, and recycling decisions.
From a review of training materials and analyses of stakeholders' perceptions, the present evaluators have determined that the Project functions, in general, in accordance with the mission and goals of the National Court Appointed Special Advocate Association (CASA), together with its state and local members. The CASA mission is to support and promote court-appointed volunteer advocacy for abused and neglected children so that they can thrive in safe, permanent homes. As stated by CASA,
child outcome goals reflect what the program ultimately hopes to achieve for the children they are serving. These goals are considered, in general, to be what is in 'the best interest of the child.' Child outcome goals focus on the placement of children in safe, permanent family homes. Outcome goals are written based on the following beliefs:
Children should be safe while in placement and after court dismissal.
Children should live in permanent, safe, family homes.
Children should spend the least amount of time under court jurisdiction as possible (Http://www.Casanet.Org/Program-Management/Evaluation/ Casa-Eff-Manual-1.Htm accessed on February 20, 2007).
Policies and Guidelines
Participant feedback included evidence of the need for formalized NDGAL Project policies and guidelines. Administrators and GALs stated:
There isn't a court rule for guardians, there should be something that sets out responsibilities, on-going training, requirements for formal training. Should be more structured, and more consistent. Rules change; they need to stay current.
Minnesota has a contract with the GALs that spells out expectations. It is formalized. The contract firms up the rules.
We have a well-prepared cadre, I feel good about their skills and experience. They are handicapped administratively in how we've allowed them to work. We need to do a lot of work with infrastructure development--formalizing rules and policies.
The Supreme Court should set policies, procedures, and expectations. The NDGAL Project was a pilot project, now it's statewide, but the formalization hasn't been done.
We need to issue reports to the public so they appreciate what GALs do. Talk about the role GALs have, helping the Child Welfare System accomplish goals. GALs need support and encouragement from the public.
Each judicial district is different in terms of policies and even terminology. I would like to have more training, communication, or something to "get us all on the same page."
Roles and Expectations for GALs
This evaluation has revealed among all participant stakeholder groups differences in perceptions of the roles and non-roles of GALs. These perceptions have resulted in discrepant expectations for the GALs and some depreciation, especially among the attorneys and CWWs, of the value and effectiveness of the NDGAL Project and of the GALs. Such stakeholder perceptions are evidenced in the following data.
Relative to the roles of the GALs, one administrator succinctly summarized stating:
GALs speak as an impartial person without the bias or agency policy or agency dictates. They are to come to the case as neutral, detached, and independent. It is their responsibility to give the court their expertise, based on their knowledge of child development and child welfare, but also on their knowledge of the individual child's need for safety, permanency, and well-being. Their role is to speak as an advocate for the child to the court.
However, the following feedback illustrates the varying roles and expectations that stakeholders perceive as the work of the GALs. Project administrators stated:
They have multiple roles: 1) being a voice for the child, speaking as an advocate; 2) being a facilitator, gathering agencies together, facilitating dialog so that as families work with multiple systems everyone is on the same page, and everyone is moving; 3) being a monitor of both parents and agencies; 4) speaking in court, in meetings, at planning sessions, at times that treatment plans are put together, visitations, any time the children's issues are being heard and addressed. Overall, their role is to be vocal in all these different kinds of areas.
Last year there was a change in Termination of Parental Rights. Guardians were not trained or involved in those termination cases previously, because attorneys handled those situations. Last year the guardians were charged with being involved in those situations. This is a very crucial time for the child's advocate to be absolutely clear in their role, to have the "moxie" to carry it out skillfully, and to assure that children receive a timely, permanent relationship.
One expectation of a judge/judicial referee was stated in this way:
My main complaint is that the guardians see the lack of compliance with a court order FIRST. They do minimal to bring it back to court's attention. Any party can return the case to court for non-compliance; the GALs have a responsibility to return a case. They worry about damaging their relationship with the other parties, but their priority is to the child. I learn about the non-compliance months later; I should have heard right away.
The following attorney and CWW statements reveal directly or indirectly these respondents' expectations of GALs:
The most valued traits a guardian can have are:
1. Speak with authority during court proceedings.
2. Convey an air of professionalism.
3. Don't interact with child in an all "warm and fuzzy" attitude.
4. Don't make false promises as to what is going to happen in court.
The worst shortcomings are:
1. GALs can be too entwined with the larger family--parents, aunts, extended family to the point that they lose perspective of serving the child. They make judgments in the family's interest rather than an independent, objective personally-based statement of what the child needs. (Not what the child wants, but that which is in the child's best interest)
2. Sometimes they get to looking out for the family, and that's not the role of the GAL. That's the role of their informal support system. The GAL should be providing a formal support primarily for the child, only secondarily for the family.
3. When GALs haven't clarified for the family what they [the GAL] do. The guardian must clarify to parents and family the guardian's role. In my experience the family and child confuse the guardians with the social workers in terms of their roles. The family and child confuse the guardian's role as being a FAMILY advocate. Guardians are losing sight of the child's best interest.
4. That guardians don't come to court with written information for the family. It should include the name and contact info for the GAL and the role, duty, and responsibility of a GAL. Clarify an additional note to the parents what they can expect from the GAL in the future. There should be two copies; one copy to the family, one copy retained by the guardian as a record.
Independent decisions are extremely important. It is very important that the GAL state clearly that their first obligation is to the child. If there is conflict between that obligation and the family's wishes, they will fulfill their obligation to the child.
At times the GALs trust Social Services records that are not always accurate. Whenever I have pointed out a concern with Social Services, the GAL has always done more investigation on the issue to be certain of the actual situation. Social services has a tendency to think opinion is fact, and the GALs need to watch for that.
Ensure that attorneys and judges recognize the importance of their [the GALs'] role in the process and routinely ask for input.
[GALs] Get your facts straight, be more appropriate at meetings (i.e., act professional), less judgmental and less "case managing." Be more aware of your role; you re not the social worker.
GALs need to remember they are not the child's social worker nor the family's social worker.
Stakeholders even suggested that the role of the GALs could be expanded:
I believe GALs could be useful in custody cases not involved with Social Services. Judges have expressed a desire to have us help them with cases, much as we do in deprivation cases and as of now, there is nothing in the Century Code to allow this.
I think the family courts could use something like the GALs. They have custody investigators or attorney GALs. Sometimes a family having a dispute doesn't need a full investigation at $2000. A GAL could do some of these cases for much less money. Judges are seeing what the GALs can do with our current cases, and they want it in other cases.
There may be other matters the GALs could handle.
As a result of this context evaluation, the evaluators have advanced the following finding and recommendation:
Finding #8
Apparently, the NDGAL Project does not have written statements of mission, goals, objectives, a strategic plan, policies and guidelines, and procedures for ongoing personnel evaluation.
Recommendation #2
That the NDGAL decision-making leaders formulate, document, and disseminate among stakeholders the following infrastructure as a context for project management and evaluation:
To study further the effectiveness of the NDGAL Project, the evaluators applied input evaluation to serve structuring decisions. In the following sections the evaluators report 1) their own feedback regarding the identity of the NDGAL decision-making leadership and 2) stakeholders' appraisal of Project resources including personnel, finances, GAL training, and project recordkeeping.
Need to Identify the Project Decision-Making Leadership
The evaluators have not found Project documentation nor other formal statements that identity the Project's primary decision-making leadership or the existence of an advisory or oversight committee or board. The decision-making roles of the Project Director, Project Coordinator, UND Department of Social Work Chairperson, and Officials in the Court Administrators Office are unclear and seemingly unspecified. Therefore, we offer the following finding and recommendation:
Finding #9
There is apparently no Project documentation that identifies the existence or identity of the Project's decision-making or advisory individual or group.
Recommendation #3
That the individuals or groups currently responsible for Project decision-making, advisement, and/or oversight take action to formalize, document, and appropriately disseminate their identity, the nature and extent of their Project responsibilities, operational procedures, and contact information.
This recommendation has implications not only for the operation of the Project but also for the purposes of public access to the formally-designated representation of the Project.
Personnel Resources
Although personnel evaluation was not an intended or implied part of this study, some stakeholders offered input that they asked be conveyed in this report:
Things are much better now than three years ago when I got here. I think that's largely due to the communication and capabilities of the new director. Brad is very available and wants to do a good job.
Brad does a very good job. He, like the rest of us, needs positive reinforcement. Can you put this in your evaluation?
Stakeholder satisfaction and appreciation for the performance of Project administrators was embodied in feedback such as the following provided by the GALs:
I enjoy that Brad [Swenson] and Lanny [Serrano] don't micromanage us. They're there, but they're not interfering. I really enjoy working with them.
Brad is doing a great job so my comments about weaknesses are not a reflection on him.
Brad is helping us move forward, but we have a ways to go.
Attorneys and CWWs did not seem to have an experiential base from which to provide feedback about the Project leadership. However, other administrators and judges/judicial referees stated:
Right now, they [Brad Swenson and Lanny Serrano] do good work, and everybody acknowledges that. But there needs to be
I think everybody's satisfied with the leaders, but I know there is interest in expanding the leadership role of the director.
Brad does good work, but my concerns are about
Project Directorship
The issue that emerged about the Project directorship was expansion of the director's roles to include: 1) representation and promotion of the Project with other organizations and state agencies; 2) data analysis and research on GAL Project effectiveness and impact; and 3) fund raising for special projects. Stakeholders from the "administrator group" provided the input on this issue:
The director needs to be a player in the state child welfare circles. Brad needs to be more available and have time to be front and center at every meeting and roundtable within the child welfare system and within the ND Supreme Court administrative community. The NDGAL Project is seen as secondary, mostly because they're off doing their good work. So nobody thinks about them during policy making and planning at the state level.
The director needs to be looking at all the data that the Project is producing to show outcomes of the GAL Project and compare them to the data outcomes needed in the Child Welfare world.
The Project has no data infrastructure to show whether it is actually effective or not. What are we doing well? How do we get better outcomes for kids? I know anecdotally that the guardians have a huge role, but I want to see the data.
The Director should have the support to go out and be looking for additional funding for special projects, special training sessions, and special pilot studies.
Based on the foregoing input evaluation, the evaluators offer the following finding and recommendation:
Finding #10
Relative to administrative Project personnel, there is a high level of satisfaction with the performance of individuals. However, there are concerns that to strengthen and improve the Project leadership, roles of the director need to expand in the following areas: 1) representation and promotion of the Project with other organizations and state agencies; 2) data analysis and research on GAL Project effectiveness and impact; and 3) fund raising for special projects.
Recommendation #4
That NDGAL Project decision makers 1) establish the desired target roles and responsibilities of the Project director; 2) determine the nature of the director's current workload; 3) analyze and resolve discrepancies, as reasonable within the limits of actual and potential Project resources; and 4) establish evaluation procedures for the directorship.
Number of GALs
The stakeholders raised two issues involving the GALs as a Project resource: the number of Gals and the length of their service on individual cases.
The judges/judicial referees were asked to indicate the level of their agreement with an item, "There are sufficient GALs to meet my caseload." Only 13 individuals responded to this item. None strongly disagreed, but two disagreed. Two indicated their neutrality; seven agreed; and two strongly agreed. The two judicial respondents and a CWW who wrote about a need for additional GALs stated:
There is no recruiting of new GALs, though I've had lip service about recruiting more. There isn't one in my county. The closest one is approximately 30 miles away. Most of them have to travel a long way to get to my court. They get paid travel time so it would cost less to have some locally, even if occasionally there is a conflict of interest.
We also need more GALs to reduce travel time and expenses. Too few are used too frequently.
We need more GALs! We only have one in our area, and I believe he occasionally can be overwhelmed and confused when multiple referrals come at once.
Addressing an ethnicity/cultural concern, two judicial officers stated that:
We need Native American GALs who are familiar with active efforts to serve Native children and who have cultural sensitivity to our North Dakota Tribes.
We need Native American guardians. There is little or no effort to recruit them. Native American guardians would be useful across the state. Guardians with cultural sensitivity could be utilized across the state.
Two CWWs also addressed this need:
About 25% of the kids are from minority groups. We need some minority GALs!
I think GALs are culturally sensitive, but it would be good to recruit Native GALs, and I even have a name that I'll pass on to Lanny.
Duration of GAL Appointments
Participants expressed much interest in and concern about the duration of the GALs' services on cases. For some respondents the issue is a question of whether some cases are being appropriately monitored. The data reveal inconsistent preferences, expectations, and practices across the Project. Only representative quotes are provided below because of the extensiveness and the redundancy in the feedback.
GALs are appointed initially for adjudication and disposition of the initial full hearing of the child. After that point in time, the Guardian is taken off the case, but somewhere 9-10 months later, if the child is still in foster care, that guardian or another is assigned to the child's case for the next court hearing. There is a gap there. While the child is in foster care, during that gap, it is absolutely essential that we have a GAL to speak for the child's best interest. We need the GALs to monitor, to facilitate, to advocate for the safety, permanency, and well being of that child. In the best and ideal world, we've got case managers that would do this. For a lot of reasons, the case managers don't recognize the need, or don't have the time, whatever it might be. The GAL is our assurance that those best interest needs are being addressed.
We can't afford to monitor all cases. We have to decide which cases the extended monitoring would be best used on. Primarily it is a financial dilemma. It's not that we don't want them to do follow-up; we just can't afford it.
Guardians should stay on the case. The court case comes up a year later, and you have to backtrack and re-research the case. They aren't saving money by taking the GAL off of the case, and the kids could use the advocacy in the meantime. I ask to stay on the case and monitor it. One judge wouldn't approve it, unless there are extraordinary circumstances, but other judges approve extra time.
When the guardians do not continue with the case, then when I want to consult with them to get their opinions, I have to break confidentiality if they aren't in that official role of GAL.
The judge assigns the guardian for a certain period. After the hearing, my job is done. No monitoring. Different judges define that term of service differently. I want more long-term contact to ascertain progress--to make sure things are going the way they should be. In some cases long term monitoring isn't needed. Here, we ask the State's Attorney to request a longer term of service.
I have a difficult case that hasn't even gotten to court yet, and I've got nearly 15 hours already. We should have the freedom to invest the time we feel is needed. I think the GAL should stay involved until the GAL decides involvement isn't needed any more. The decision to continue monitoring should be made by the GAL not the judge.
The Judge appoints me and tells me "I'm on the case until it's resolved." I know in six months or a year later, I won't have to spend a bunch of hours getting up to speed. It's a matter of cost effectiveness.
Monitoring after the court decision is case-by-case--depending on whether the case is likely to continue. It's more cost effective to let us monitor than to play catch-up later.
I think that we could better serve children by staying involved in the cases past disposition. We often get re-assigned to a case weeks/months later, and it takes a lot of time and effort to update information and re-assess the status of a case.
I feel it is more effective to stay on cases after the county gets custody to monitor and advocate for permanency for the children.
Used to be a GAL was appointed once a kid was in the system. Then when the court order came down, the GAL was done. Now GALs can continue to monitor at the discretion of the judge, but here we aren't doing this. It costs too much. If a guardian stays involved to monitor, even if they just check on things one-two hours a month, is that more cost effective than to be re-assigned during a permanency hearing? Is it in the child's best interest to have them stay involved?
Stakeholder feedback supports the following findings and recommendation:
Finding #11
The data do not clearly support the need for additional GALs except for the concern that Native American GALs are needed in some areas of the State.
Finding #12
The data reveal inconsistent preferences, expectations, and practices across the Project relative to the duration of the GALs' service on cases raising concerns about whether cases are being consistently, appropriately, and economically monitored.
Recommendation #5
That the NDGAL decision-making leaders 1) formulate, document, and disseminate a Project policy regarding duration of GALs' service on cases including procedures for appeal and/or exclusion; and 2) adopt procedures for determining the impact of the policy on "children's best interests" and Project funding.
Fiscal Resources
Relative to the fiscal resources of the NDGAL Project, the key stakeholder concerns were 1) funding for the project; 2) administrative costs of the Project; 3) the limit on the number of billable hours; and 4) reimbursement for work-related expenses incurred by the GALs.
Administrators and a judge addressed project funding and administrative costs:
Systemically we have a couple of challenges. The Project is greatly under-funded and understaffed. It's run on a shoestring.
I'm concerned about the cost of administration. The grant money hasn't increased, but salaries and everything else have. It's already costing more than what the grant money provided. It's being supplemented by the general fund, but this needs to be looked at. The money has to come from somewhere.
I'm concerned about the financial aspect. This is all funded by a Federal Court Improvement Grant. But they spend 130% just to administer the program. That doesn't include the pay of the guardians, which comes out of the General Fund. They get $100,000, and we still have to go to the legislature to get General Fund money for the remaining 30%.
GALs and CWWs advanced their concerns about the limit on the number of billable hours and unreimbursed expenses for GALs:
I would never be able to bill for all the clock hours I devote to each case. As it is, I seem to almost always exceed the number allotted for each case.
Regarding the clock hours allowed per case, that's a problem. It varies so much as to number of hearings involved, number of children in the family, geographic area and distances. The system really needs to be more flexible.
GALs need to be paid more. There is one raise after three years, and then no more. There are many things you don't bill for--phone calls, mail and, travel--but they all add up.
One of our GAL's complaints is that she doesn't get reimbursed for travel to see the people involved in the case--the children, parents, etc.
Recordkeeping
Although there was little discussion of recordkeeping, the following input offered by administrators merits consideration.
Another thing is just the way we keep records. A lot of it is handwritten in 3-ring binders. It's cumbersome. Timesheets and other administrative statistical data gathering have gotten better, but there's still room for improvement.
The way we get information from the guardians is very time consuming. If it were done as part of a software program, it would be easier to develop the statistics that the Administration Office needs.
Technologically, the Project is way behind. If the data could be recorded and accessed better electronically, it could be better used to determine project effectiveness to support grant requests.
Finding #13
Non-personnel resource issues addressed by stakeholders included: 1) funding for the project; 2) administrative costs of the Project; 3) the limit on the number of billable hours; 4) reimbursement for work-related expenses incurred by the GALs; and 4) non-electronic data recording and analysis.
Recommendation #6
That the NDGAL Project decision-making leaders continue to seek increasing fiscal support for the Project from both traditional sources as well as new sources to supplement the traditional sources (e.g., benefactors, sale of educational services, and interagency cost-sharing).
Effectiveness of the GALs
In general, process evaluation addresses issues such as program procedures, program activities, and personnel training. In this study the two main process-related topics were effectiveness of the GALs and the GAL training. In this section on the effectiveness of the GALs, three areas of GAL performance and activity are evaluated: 1) GALs' understandings and work performance; 2) GALs' contacting and interviewing various parties; and 3) GALs' case-related activities.
GALs' Understandings and Work Performance
The four questionnaire respondent groups used a Likert scale--1 = Strongly disagree, 2 = Disagree, 3 = Neutral, 4 = Agree, 5 = Strongly agree to indicate their level of agreement with statements about the six types of GAL understanding and work performance presented in Table 10. From 77-88% of the respondents agreed or strongly agreed that GALs understand the Court and Child Welfare Systems, carry out their work professionally and objectively, ask pertinent questions, and make appropriate recommendations. The highest percentages of disagreement plus neutrality ratings combined were 23% on the items about GALs' understanding of the Child Welfare System and their objectivity in carrying out their work.
Table 10. Number and Percentage of Respondents Indicating Levels of Agreement with Selected Statements about the Understanding and Activities of GALs (1 = Strongly disagree; 3 = Neutral; 5 = Strongly agree).
| To what degree do you agree or disagree with the following statements about the | 1 | 2 | 3 | 4 | 5 |
| GALs understand the court system. |
| # of Respondents (N = 116) | 0 | 3 | 15 | 71 | 27 |
| % of Respondents | 0 | 2.6 | 12.9 | 61.2 | 23.3 |
| GALs understand the child welfare system. |
| # of Respondents (N = 115) | 0 | 4 | 22 | 63 | 26 |
| % of Respondents | 0 | 3.5 | 19.1 | 54.8 | 22.6 |
| GALs carry out their work professionally. |
| # of Respondents (N = 116) | 0 | 5 | 8 | 58 | 45 |
| % of Respondents | 0 | 4.3 | 6.9 | 50.0 | 38.8 |
| GALs carry out their work objectively. |
| # of Respondents (N = 115) | 0 | 10 | 17 | 54 | 34 |
| % of Respondents | 0 | 8.7 | 14.8 | 47.0 | 29.6 |
| GALs ask pertinent questions. |
| # of Respondents (N = 116) | 0 | 3 | 12 | 61 | 40 |
| % of Respondents | 0 | 2.6 | 10.3 | 52.6 | 34.5 |
| GALs make appropriate recommendations. |
| # of Respondents (N = 113) | 0 | 1 | 18 | 55 | 39 |
| % of Respondents | 0 | .9 | 15.9 | 48.7 | 34.5 |
In Table 11 a mean of 3.5 or higher is a measure of agreement with an item. The mean ratings for all groups ranged from 3.5 to 4.0. As shown in Figure 5, the attorney's and CWW's mean ratings were relatively lower than those of the GALs and judges.
The content of the stakeholder feedback quoted in this section was generally reported by only one to three individuals about one or two GALs. However, the data address issues and concerns of importance to the long-term quality and effectiveness of the NDGAL Project. Further, appropriate administrative actions may be elicited by such findings (e.g., topics may be noted for future training sessions or the bases for routine personnel performance evaluation may be discovered).
Table 11. Mean Responses of the Four Participant Groups Indicating Levels of Agreement with Selected Statements about the Understanding and Activities of GALs (1 = Strongly disagree; 3 = Neutral; 5 = Strongly agree).
| To what degree do you agree or disagree with the following statements about the | Judges | Attorneys | CWWs | GALs |
| GALs understand the court | 4.1 | 3.7 | 4.0 | 4.2 |
| GALs understand the child | 4.2 | 4.0 | 3.8 | 4.2 |
| GALs carry out their work | 4.4 | 4.2 | 4.0 | 4.7 |
| GALs carry out their work | 4.1 | 3.5 | 3.8 | 4.5 |
| GALs ask pertinent questions. | 4.3 | 3.9 | 4.0 | 4.6 |
| GALs make appropriate | 4.1 | 3.9 | 4.0 | 4.8 |
Figure
5. Mean Responses of the Four Participant Groups Indicating Levels of
Agreement with Selected Statements about the Understanding and Activities of GALs
(1 = Strongly disagree; 3 = Neutral; 5 = Strongly agree).
Some respondents highlighted the difficulty of providing perception data that could be generalized to an entire group:
This survey was difficult to complete as the capabilities and areas of strength vary greatly from one GAL to the next. We have a few GALs that are not practicing healthy boundaries with families, are not presenting themselves professionally, and are simply regurgitating the information they read from the CPS and wrap around files. Other GALs are very professional, maintain appropriate boundaries, and meet with family members and collaterals extensively and, therefore, provide additional information that helps in providing services to the families.
Have sometimes found GALs (not all) to be very direct to the point of upsetting a child/family, making the process difficult, almost to the point of antagonism.
It varies greatly depending on which GAL is assigned to the case. I enjoy working with many of the GALs who come prepared knowing the case. Some GALs appear to be unprepared and not having met the child. Due to my job roles and responsibilities, I rarely see a GAL report.
Some GALs are over-involved, others are scattered and under-involved.
Our main GAL has a great relationship with all parties. Another GAL kind of makes up her mind before she has any input, before she talks with the kids... We actually had a difficult case where we had to ask her to defer to the other GAL.
Nonetheless, the qualitative data offered insight into some of the participant's reasons for their ratings. Regarding the GALs' understanding of the court system, CWWs offered their perceptions:
[GALs need to] learn the Child Welfare System better. One GAL always gives me the same statement each time she's assigned to a foster care child: "This child can never go home." This statement tells me the GAL does not understand the reunification process. GALs need to understand "reasonable efforts" and not offer TPR right away.
One GAL in particular does not understand the Child Welfare System and the role and responsibilities of Social Services. Before she's even met with any of the family, she says that parental rights need to be terminated. This is not helpful to anyone and is surely a very serious judgment to make about a child/family you've just been assigned to.
Respondents offered the following feedback relative to GALs' carrying out their work professionally.
I take confidentiality seriously. Living in the community, I hear things. Things get "chatty," and people know too much that they shouldn't know. Perhaps the GALs are leaking info; perhaps it's someone else. We have to be more careful about what is said, and where it's said. Don't do business at a ball game or cafeteria.
Not be so gossipy; stick to just the case.
Respondent reactions to "GALs carry out their work objectively" included the following comments by attorneys:
I feel that on some occasions, the GAL appears to be just an extension of Social Services and may be too influenced by the Social Services case worker. Maintaining an objective point of view is important in every case.
They always seem to be aligned with social services. They need to be more neutral in the process. Their reports often parrot the Social Services report and don't include a lot of independent thought or investigation. I never look at them as a potential source of information for me as the defense attorney.
Their initial thought is always removal. Perhaps a more open-minded approach should be championed.
On rare occasions a very few GALs have become "vested" in the outcome, thereby being an impediment, or (even worse) becoming still another adversary.
Most GALs in our area of the state are very judgmental against parents.
CWWs added:
They need to gather objective information from many sources in contact with the child.
Not to get emotionally involved. Sometimes it seems they are wanting the kids to remain with the foster parents because they have a close relationship with those foster parents. We have to go by minimal standards when reuniting a child with their parents.
Some GALs seem to favor or be more involved in one case over another--there is more contact, stronger advocacy, more recommendations.
Although feedback already discussed in other topical areas of this report addresses the issue of the appropriateness of GALs' recommendations, the following statements by CWWs and GALs merit attention because of additional issues related to GAL recommendations:
Be able to provide strong testimony regarding their contacts on a case and the recommendations to the court.
GALs have, at times, made recommendations based on attending a single meeting.
All GALs should interview the children prior to reports being written and recommendations being made. Comments/recommendations should not be made without their direct knowledge of a case/child's situation. For example, recently a GAL stated in a family meeting that a child was doing extremely well in a foster home when, in fact, the child wasn't, and we were trying very hard to get him out of the foster home.
GAL recommendations need to be presented to the courts just like Social Services' case plans are.
[I wish] That the decisions made in SPOC (perm plans) would be upheld by the courts and not undermined and a secondary decision passed on to the court.
Here in [larger city] we have these meetings--Child and Family Team meetings. Everyone is involved--parents, foster parents, the caseworker, the GAL, etc. And we decide on recommendations. The next day Social Services has a "staffing meeting" and changes the recommendation. People who aren't even on the team for the child make the secondary decisions.
GALs' Contacting and Interviewing Various Parties
Using a scale ranging from 1 = Not effective; through 3 = Somewhat effective; to 5 = Very effective, the four groups evaluated the effectiveness of GALs in contacting and interviewing various parties. As shown in Table 12, from 83-94% of 104 or more respondents provided ratings of somewhat effective to very effective. The highest percentage of low (1 or 2) ratings were directed at GALs' contacting and interviewing other relatives. The mean responses reported in Table 13 and displayed in Figure 6 show the lowest mean ratings to be those of the CWWs.
Table 12. Number and Percentage of Respondents Indicating Levels of Effectiveness of GALs in Contacting and Interviewing Various Parties (1 = Not effective; 3 = Somewhat effective; 5 = Very effective).
| How effective are GALs in contacting and interviewing various parties in support of | 1 | 2 | 3 | 4 | 5 |
| Children/youth |
| # of Respondents (N = 113) | 1 | 3 | 24 | 35 | 50 |
| % of Respondents | .9 | 2.7 | 21.2 | 31.0 | 44.2 |
| Biological parents |
| # of Respondents (N = 113) | 1 | 3 | 26 | 39 | 44 |
| % of Respondents | .9 | 2.7 | 23.0 | 34.5 | 38.9 |
| Foster parents |
| # of Respondents (N = 107) | 1 | 9 | 25 | 40 | 32 |
| % of Respondents | .9 | 8.4 | 23.4 | 37.4 | 29.9 |
| Other relatives |
| # of Respondents (N = 104) | 3 | 14 | 31 | 36 | 20 |
| % of Respondents | 2.9 | 13.5 | 29.8 | 34.6 | 19.2 |
| Collaterals (e.g., teachers, doctors, neighbors) |
| # of Respondents (N = 110) | 2 | 14 | 30 | 35 | 29 |
| % of Respondents | 1.8 | 12.7 | 27.3 | 31.8 | 26.4 |
Table 13. Mean Responses of the Four Participant Groups Indicating Levels of Effectiveness of GALs in Contacting and Interviewing Various Parties (1 = Not effective; 3 = Somewhat effective; 5 = Very effective).
| How effective are GALs in contacting and interviewing various parties in support of | Judges | Attorneys | CWWs | GALs |
| Children/youth | 4.5 | 4.4 | 3.9 | 4.4 |
| Biological parents | 4.2 | 4.2 | 4.0 | 4.3 |
| Foster parents | 4.0 | 4.0 | 3.7 | 4.2 |
| Other relatives | 3.7 | 4.0 | 3.3 | 3.8 |
| Collaterals (e.g., teachers, | 3.9 | 4.3 | 3.3 | 4.2 |
Figure 6. Mean Responses of the Four Participant Groups Indicating Levels of Effectiveness of GALs in Contacting and Interviewing Various Parties (1 = Not effective; 3 = Somewhat effective; 5 = Very effective).
This topic generated an abundance of qualitative data. Relative to GALs' contacting and interviewing children and youth, the following feedback is representative of many comments by CWWs and GALs:
More face-to-face contact with children.
Have more contact with kids.
Monitor the case better, not just right before hearings.
It would help if GALs met the children before the day of court. Also if they observed them with parents and foster parents.
To spend more time with children and their families--get involved more quickly/actively. Increased involvement during entire placement and not just with court hearings and special circumstances,
Meet with the children more frequently. Many times they attend meetings and court hearings and have never met the children but try to give them advice or tell them what to be doing. The kids lose respect for their GAL and make comments like, "She doesn't even know me."
Have more contact with the children not in foster care placements. I have worked with the program through foster care and family preservation, and the amount of contact with the family preservation cases is not comparable.
GALs are at a disadvantage when there is a shelter care situation. Often they have not met with child/parents due to the short notice of the hearing and their appointment. It would be nice if they had the ability to meet with clients sooner.
GALs attend supervised visits with the child and family. It really shows them how these families interact, and I feel like they've made an effort to know the family and children they are working with.
I feel bad for them; they get called in on a moment's notice. It's a pretty tough position to be in.
Our court won't establish a shelter care hearing until the GAL is assigned. The guardians have to drop everything to get to the hearing. They're certainly dedicated to their jobs.
GALs should be allowed more time with children. The courts could designate that more time be spent by GALs with children.
[GALs see] the condition of the home, interaction between parent and child; such as checking to see if child evades parent or guardian.
Maybe remember to contact child periodically between children and family team meeting.
A judge, CWW, and GAL offered additional general comments addressing GALs' interview/contact-related activities:
It's useful in court when GALs have talked to extended family to have them help with children; visited the at-risk family in the home and made observations; and have spoken with a neighbor and with the children's teachers for additional fact-finding.
Encourage them to have more contact with the parties involved in the case. Perhaps they need more training on how important that contact is.
If Social Services has an adversarial relationship with the family, sometimes we can run interference with the family--tell parent's "it's in your best interest to do this" Sometimes Social Services would want us assigned longer. Social Services can play the role of "Bad Cop," and we play "Good Cop".
GALs' Case-Related Activities
Using the same five-point effectiveness scale, 104 or more respondents rated GAL effectiveness with eight case-related activities. As shown in Table 14, from 90-97% of the ratings were Somewhat effective to Very effective (ratings = 3-5) with 7-27% of the respondents' ratings Somewhat effective. It is reassuring to note that 101 of 113 (approximately 90%) of the respondents rated the GALs' "considering the best interests of children (e.g., advocacy)" as a 4 or 5--in the very effective range. The item receiving the greatest percentage (approximately 40%) low ratings (1-3) was "assisting with permanent placement for the child/youth." However, as shown in Table 15 and Figure 7 the mean ratings of GALs' effectiveness on all eight case-related activities were 3.5 or higher on evaluation of effectiveness provided by all four respondent groups.
An extensive amount of qualitative data addressed these eight GAL case-related activities. Again, this type of data provides a view into the reasoning underlying the quantitative ratings. Regarding the researching of case facts, attorneys, CWWs, and GALs provided the following perceptions:
I have found that the GALs rely on the information generated/compiled by Social Services which can lead to parents believing that the GAL is just another social worker there to take their kids away.
The GALs are just a parroting of what Social Services says. They look at a Social Services' report and just back it up. There has to be some way to make the program more independent. My clients--the parents--just think the GAL is part of Social Services, and they're not getting a fair shake. I'm sure part of it is time. The GALs don't always have time to do the investigation that should be done. I also think it's training.
The GALs that I have worked with tend to do reports last minute and rely on county information more than information/contacts from families and children. I have not known a GAL to contact schools, therapists, etc.
There needs to be a change. At the time of initial placement, GALs have a very limited time frame to interview parents and children (sometimes less than 24 hours).
Table 14. Number and Percentage of Respondents Indicating Levels of GAL Effectiveness with Case-Related Activities (1 = Not effective; 3 = Somewhat effective; 5 = Very effective).
| How effective are Gals in carrying out case-related activities? | 1 | 2 | 3 | 4 | 5 |
| Researching case facts |
| # of Respondents (N = 113) | 1 | 6 | 25 | 43 | 38 |
| % of Respondents | .9 | 5.3 | 22.1 | 38.1 | 33.6 |
| Considering the best interests of children (e.g., advocacy) |
| # of Respondents (N = 113) | 0 | 4 | 8 | 41 | 60 |
| % of Respondents | 0 | 3.5 | 7.1 | 36.3 | 53.1 |
| Preparing and maintaining appropriate records |
| # of Respondents (N = 104) | 0 | 9 | 21 | 42 | 32 |
| % of Respondents | 0 | 8.7 | 20.2 | 40.4 | 30.8 |
| Preparing information for Court (e.g., reports, oral testimony) |
| # of Respondents (N = 114) | 0 | 5 | 19 | 42 | 48 |
| % of Respondents | 0 | 4.4 | 16.7 | 36.8 | 42.1 |
| Testifying orally to the court |
| # of Respondents (N = 113) | 3 | 8 | 20 | 35 | 47 |
| % of Respondents | 2.7 | 7.1 | 17.7 | 31.0 | 41.6 |
| Working with others in the court system |
| # of Respondents (N = 110) | 0 | 3 | 14 | 45 | 48 |
| % of Respondents | 0 | 2.7 | 12.7 | 40.9 | 43.6 |
| Monitoring the case |
| # of Respondents (N = 109) | 0 | 11 | 29 | 38 | 31 |
| % of Respondents | 0 | 10.1 | 26.6 | 34.9 | 28.4 |
| Assisting with permanent placement for the child/youth |
| # of Respondents (N = 107) | 2 | 12 | 29 | 36 | 28 |
| % of Respondents | 1.9 | 11.2 | 27.1 | 33.6 | 26.2 |
Table 15. Mean Responses of the Four Participant Groups Indicating Levels of GAL Effectiveness with Case-Related Activities (1 = Not effective; 3 = Somewhat effective; 5 = Very effective).
| How effective are Gals in carrying out case-related | Judges | Attorneys | CWWs | GALs |
| Researching case facts | 4.1 | 3.7 |