Support the Enlisted Project
Project Impact Report
Families, Communities, and Nation:
Measuring the Impact of
Support the Enlisted Project
From January 2016 through August 2016, Support the Enlisted Project (STEP) participated in a multi-phase research workshop called Project Impact. The eight-month program offered training in innovative data collection techniques specifically calibrated for non-profit and community-based organizations. STEP and the Project Impact team worked closely and collaboratively to develop a comprehensive research program to assess the quality of the impact we are having for our clients and their families and communities. This report summarizes our efforts and outlines our main findings.
Tony Teravainen, Co-founder and President/CEO
Kathi Bradshaw, Vice President and Director of Client Services
Erica Bender, Volunteer and Ph.D. Candidate at UC San Diego
Our team worked hard to make this research project a success, but we would not have been able to capitalize on these efforts if it weren’t for the hard work and generosity of many others. Specifically, we would like to thank:
The Gumpert Foundation, The Leichtag Foundation, Shelby Ross, Steve McClelland, STEP Board of Directors, and STEP’s former clients who agreed to be interviewed and/or returned our questionnaire. We would also like to thank Steve Patty and Jessamyn Luiz for their support and guidance through this process.
Support The Enlisted Project (STEP) assists junior active duty enlisted members and recently discharged enlisted veterans and their families in Southern California facing financial crisis achieve long term financial self-sufficiency through counseling, education and grants to alleviate critical near term obligations. STEP provides its services to the lower six paygrades (E1-E6) of our military and veterans within the first 18 months after their honorable discharge. Southern California is home to 95,000 active duty military in the E1-E6 pay grades, of which half have families. 60% of these military members are Low Income per HUD Standards; 30% rely on food assistance regularly to feed their families; 73% are under age 26. These families are also without the support systems of their hometowns as they were sent to Southern California by the military.
STEP’s primary goal is to prevent these young military and veteran families from loosing one of their basic life necessities due to an accute financial crisis, and to build them towards a path of financial self-sufficiency. Everything STEP does works toward this sigular, critical goal. STEP provides emergency financial counseling and education in a one-on-one forum with a case worker who is also a social worker. This counseling and education has a goal to empower each client to take ownership of their finances, produce a monthly budget that works for their unique situation, and to produce a long-term debt reduction plan. When necessary, STEP will also provide a grant, not a loan, that will prevent the loss of the family’s basic needs while the family gets back on their feet. Additionally, STEP maintains a warehouse where families can visit and pick up donated goods, such as baby supplies and home goods, and offers cost-ofsetting family programs that help families save money, thus increasing their ability to maintain their budgets or increase savings.
STEP relies completely on privately donated funds to execute its mission. Since 2012, STEP has raised and spent more than $3 million to assist 2,500 young, military families out of a verifiable financial crisis and onto a path of financial self-sufficiency, issued $1.1 million in grants to lenders and creditors on behalf of our young military families, and distributed $600,000 of donated goods and services to an additional 5,000 miliary family members. Our grants ensure that our military families do not have to go without a basic need while they are getting their financial situation back on track. Our counsleing empowers them to take ownership of their financial future, giving them a sense of ownership and independence on their financial path. STEP gives each family requesting assitance personalized financial counseling and education which they need to survive over the long run, as well as the immediate relief from an eminant financial crisis. 89 percent of families who receive a grant from STEP do not return for a second allowed assist.
The Emergency Financial Assistance (EFA) program is the backbone for all services provided at STEP. The EFA program provides financial counseling and, when necessary, financial grants to military servicemembers and veterans who are currently experiencing a financial crisis. Young military personnel and their families face many of the same financial management challenges as civilians. Like most young adults, they must continually grapple with everyday financial decisions like budgeting and bill paying while also managing long-term considerations like retirement planning and debt management.
However, being in the military also involves unique circumstances and stresses that can make financial management even more difficult. For active duty military families, the frequent moves, the variable availability of Base housing, the difficulty spouses face in finding employment after each move, and the relatively low pay in the junior enlisted ranks combined with Southern California’s high cost of living, all create additional financial strain. Frequent family moves are often overlooked as a major souce of financial stress, but STEP client data shows that of the EFA applicants, 65% have moved within the past 24 months, 42% have moved within the past 12 months, and 25% have moved within the past 6 months.
Nationally, nearly 50 percent of veterans try to stay in the city in which they discharge from service. These veteran families, facing the difficult process of transitioning to civilian life, find that Veteran Benefits and G.I. Bill stipends are delayed, unemployment is high, and spouse employment is still very difficult. When these factors combine with the high cost of living in San Diego, young military and veteran families can easily find themselves in a financial crisis. In Southern California, there are 25,000 service members that transition to become veterans each year. Los Angeles and San Diego rank 1st and 3rd respectively in veteran population, and San Diego County has the highest concentration of veterans under the age of 25 in the nation.
STEP’s EFA program is designed to assist these families through counseling and grants, not loans. The grants issued never need to be repaid by the families; in fact, all STEP services are free to its clients. STEP works with the family to review their budget and help determine how and why the crisis occurred and get each family on a positve budget path. Then, if necessary to restore or preclude the loss of a basic necessity, STEP issues payments directly to the landlord, utility company, bank, or other appropriate entities. About half of all EFA clients will require, and be issued a grant. STEP also works closely with local mechanics to help ensure families have reliable trasportation, and issues gift cards to local grocery stores or the Base Commissary to ensure families can pay for food. Through these direct payments totalling over $1.1 million, STEP’s Emergency Financial Assistance program helped families meet a number of basic needs:
STEP also helps with vehicle insurance, critical baby items, dental emergencies, emergency travel and other financial hardships that put families in a financial crisis.
The intended impact of the EFA program is that military and veteran families will move from financial crisis to financial self-sufficiency through a combination of counseling, education, and resources that help them pay their bills and/or offset their costs of living.
Because financial self-sufficiency is the singularly most important intended impact for STEP, we focused all of our research efforts toward determining whether our client families had achieved some level of self-sufficiency after receiving services from STEP. We recognize that a client has achieved financial self-sufficiency when they are independently capable of meeting their financial needs, accept ownership for their financial situations, and take specific steps to improve their situations.
We developed a number of indicators that would reveal whether our clients had achieved financial self-sufficiency. Using the Dialogues In Action “Heart Triangle” model we identified indicators that would reveal both surface-level changes as well as deeper, life-altering transformation. We then studied the extent to which our clients had achieved those indicators of impact. Our indicators of impact were as follows:
Both our qualitative and quantitative protocols were designed with these indicators and our overall intended impact of financial self-sufficiency in mind. Our qualitative interview protocol and survey instrument can be found in the Appendix.
We used a combination of qualitative and quantitative data to assess the quality of our impact and the extent to which clients demonstrated any of the six indicators of impact above. We utilized a short, quantitative survey instrument to assess whether former clients reported any surface-level emotional or behavioral changes since working with STEP (Indicators 1-3). We utilized in-depth qualitative interviews in order to understand former clients’ current financial lives and whether any lasting transformations had occurred with respect to their financial self-sufficiency (Indicators 4-6).
We aimed to complete 15-18 interviews with clients along three different sampling criteria: a combination of both active duty and veteran clients, married and single clients, and clients who had receieved a grant and counseling under the EFA program as well as clients who had received counseling only, without a grant. We generated a random list of clients who had received our services at least four months but not more than twelve months in the past, and contacted them by phone to ask for interviews. Like most of our service interventions, all of our interviews were conducted by phone. The interviews typically lasted between 30 and 45 minutes.
We immediately faced an issue with connecting to clients. Former clients were not likely to take our call, and we left several unreturned voicemails. When we did speak with former clients we were delighted that many were willing to be interviewed. However, it was difficult to ensure that the client would follow through with their scheduled interview time. In other words, it was difficult to connect with former clients at all, let alone multiple times to schedule and conduct our interviews. There are several potential reasons for this disconnect, including the fact that STEP works with a population that is often moving around the country and changing their contact information. Despite these challenges, we were able to garner a sample of 8 in-depth interviews with former clients.
We devised an anonymous, nine-question electronic survey that we sent to every client whom we had assisted from November 2015 through May 2016. The survey was intentionally short; we wanted to assure former clients that participating in the survey would be fast, user friendly, and painless. We sent the survey to clients for whom at least four months had transpired since receiving services from STEP. This was to mitigate potential biases that may result from having worked with STEP too recently, including the emotional (positive or negative) feelings toward a particular case manager or staff and the potential fallout (positive or negative) from receiving a grant. We decided that four months was an appropriate amount of time for our former clients’ lives to return to normal, thereby providing a window to see our impacts.
We distributed our survey electronically using the iContacts software tool. We sent email notifications with links to the survey three times. We received 17 responses, with a response rate of seven percent. This response rate is consistent with other follow-up surveys that we have sent to clients in the past.
Before reviewing our findings from the qualitative and quantitative data, it is important that we highlight our Theory of Change in order to put our findings in perspective. STEP utilizes a number of social work principles and theories both in the design of our program as well as in the implementation of our procedures and in our interactions with clients. We operate on a social work model that utilizes a solution-focused approach with an emphasis on clients' strengths. The solution-focused approach is an evidence-based practice in the field of social work. The foundation of the solution-focused approach is that the best clinical interventions are relatively brief and goal-oriented in ways that build on the client’s strengths. It focuses on what the client wants to achieve, previous successes they have had, and immediate steps they can take in the future to achieve their goal.
Many of our findings are consistent with the solution-focused model that we have built into our program and case management procedures.
In addition, we also have five principles behind our practices that inform our programs and our choice of the solution-focused approach. Those are:
We were happy to discover that most of our findings, even the most surprising ones, were somehow connected to the solution-focused approach or the theory of change principles that STEP has intentionally built into its program.
Several factors limit our ability to generalize our findings to make broad and conclusive statements about STEP’s impact. These include:
a) Small sample sizes and low response rates: the lack of interview and survey responsiveness means that we have derived our findings from a very narrow proportion of the population with which STEP has worked in the past year.
b) Interview versus survey responses: because the survey responses were anonymous, we have no way to know who took the survey and whether there was overlap in these respondents. However, due to our small sample sizes, overlap is possible but not probable.
c) Selection biases: Our sample has two potential selection biases:
d) Team limitations and time constraints: due to limitations in manpower and resources, much of the data collection was conducted by one team member, and thus there may be some potential researcher bias in the interpretation of the data.
Despite the limitations of our data, we found seven remarkable trends among our former clients. These findings communicate to us what we are doing well and how STEP might improve as we continue to grow and refine our program into the future. We are also reminded about the pressing external factors that take a significant toll in the lives of our active-duty and veteran families.
Our difficulty in reaching our former clients was the first indicator that clients may not feel very engaged with STEP and therefore do not maintain or seek to maintain contact with us after they have received assistance. The low response rate in both the survey and our requests for interviews communicate that clients may not feel a strong emotional relationship with STEP. In addition, every client who was interviewed was unable to name the case manager with whom they worked when they went through STEP’s Emergency Financial Assistance program.
This finding is significant because it forced us to evaluate whether we found the lack of engagement acceptable as we move forward as an organization. On one hand, our focus on brief, solution-focused interventions do not encourage repeated follow-ups nor sustained connections with clients; we want the clients to get what they can from us and move on with their lives. On the other hand, we want clients to know that they can still reach out to us in the future if they want to receive financial counseling due to a change in circumstances or a desire to re-work their budgets. This finding illuminates that we can and should be more aware about the level of engagement we want and expect from former clients so that we can set clearer goals for the organization as it grows.
Virtually all of our interview respondents stressed how helpful STEP was in helping them during what was a very stressful and fearful moment of their lives. Many of them indicated that without the financial assistance in paying a particular bill or expense, they don’t know what they would have done or where they would have turned.
I just wanted to let you know I am very appreciative of the STEP program and the ease to get help. When I went to other organizations for help, they weren’t as friendly with me.
We really got ourselves into a bad spot and you guys were there for us in a way no one else was. It was really good timing because we were, like literally, running out of money.
Well you guys helped us a lot with that grant, so we were able to have one less thing to worry about. I think that helped us focus on other things to pay down and then it was just good to have less stress for a while.
This was one of the most encouraging trends in our data. STEP’s goal is to help east the financial strain of military and veteran families who are facing an imminent financial crisis and to help them move toward financial self-suffienciency. This finding reveals that we are meeting our goal of assisting them in crisis and, when combined with our other findings, suggest that it is a crucial component of building financial self-sufficiency in our clients.
One of our indicators of impact would be that clients would feel reduced financial stress as a result of working with STEP. This indicator was strongly confirmed in our survey data.
We asked survey respondents to rate their level of financial stress before working with STEP on a scale from zero to five, with zero indicating “no stress” and five indicating “very high stress.” None of the survey respondents rated their “before” level of stress as a zero, one, or two (no stress, very low stress, or some stress); everyone rated their “before” stress level as a three, four, or five (moderate stress, high stress, very high stress). Over half of our respondents reported their “before” stress level as a five – very high stress.
When we asked survey respondents to rate their financial stress now as a result of working with STEP, we saw a drastic reduction in the reported levels of stress. Whereas not a single respondent rated their “before” stress level as a zero, one, or two, over half of the respondents rated their “now” stress level in these categories. While half of our respondents reported their “before” stress level as a five – very high stress, none of these respondents rated their current stress levels as high.
The mean “before” stress level for our survey respondents was a 4.18 out of 5. The mean “now” stress level was a 2.29 out of 5. These survey responses indicate that STEP was able to reduce clients’ financial stress by 45%. It is difficult to rationalize a 45% stress reduction, but when utilizing the scale used in the survey, we find that the mean stress levels were reduced from between High and Very High to between Some and Moderate. This is a significant finding not only because it helps us confirm that we are achieving one of our indicators of impact, but because it gets to the heart of why STEP was founded in the first place.
A surprising finding that challenged our assumptions came through our qualitative interviews. During these discussions, many of the clients we spoke with offered up sincere gratitude for the assistance they received and talked about how their current financial situations had improved in the time since they originally came to STEP. However, these same clients did not attribute their reduced stress or improved financial state to the financial skills or budget help they received at STEP. Many could not even recall receiving these kinds of counseling services from STEP at all.
I don’t really remember learning anything in particular. It was just such a relief that we got a grant that we didn’t have to pay back.
I don’t remember the education. I found you guys through Google because we were living in a house that was not financially responsible and we got behind on our rent, and you guys got us squared up on that. But in terms of teaching or training I didn’t really get any of that.
While many clients did not explicitly credit STEP with their reduced financial stress or more stable financial life, we did see a curious subtext in the interviews that we found interesting. The same respondents who claimed that STEP was not the most important factor in improving their financial life were also talking about their budgets, debt-reduction, and saving plans in very specific and concrete terms, using the same or similar goals and benchmarks that STEP uses in its financial counseling. When most clients come into STEP, they claim to have budgets but to not use them or even know where to find them, or claim to want to save but to not know how. By contrast, our interview respondents could offer relatively concrete and coherent strategies that they are using to achieve greater financial success. For example, respondents talked about how they sit down with their spouses to work out the budget, many of them talking about the specific time period, frequency, and strategies that go into their budget planning. Another respondent was able to offer a very precise benchmark for his emergency savings fund – $10,000 – the same figure that STEP uses when teaching about emergency savings funds. Other respondents used precise language as well:
Well we are paying our bills on time, which is good. We keep an electronic budget, we just put everything into an Excel spreadsheet. So my husband and I both have fixed paychecks so we put everything in and see where the money is going, and we decide how to pay down debt and save, and we make sure we aren’t maxing out on anything.
Well, we have learned that it is pretty important to have a savings account set aside especially when going underway or deploying. The lady my wife talked to when this whole thing happened suggested we needed to have like $13,000 set aside for emergencies. So, that’s definitely the goal, but it’s pretty hard to obtain.
Unfortunately, our survey data cannot speak to this trend in the interviews. While, most of our survey respondents agreed that they had learned at least one skill for improving their finances, we did not explicitly ask survey respondents whether or not learning those skills contributed to their reduced financial stress.
While we do not mean to imply that our clients are wrong in not crediting STEP, nor to put words into their mouths by pointing out the things that were implied but not said, this finding is still significant to us. At minimum, this finding is instructive for how we should be engaging and interacting with clients during and even after giving them financial counseling. We want to be making sure to drive home the specific and concrete goals with our clients because that is what they are most likely to remember. We also want to reinforce to them several times how much they have learned and the growth they have been able to achieve in our program, so that they can remember not just that STEP helped them, but also how STEP helped them.
Both the interview and survey data confirmed that virtually all of our respondents are focusing on a specific skill or habit to improve their financial situation and that they have a clear financial goal toward which they are working. Many of our interview respondents discussed a desire to spend less, and cited specific areas where they have reduced their spending.
I live with the bare minimum now. If I don’t need something, I won’t get it. I also cut down on things for a while, like cable, until I was able to afford it.
I think we are just trying to be more aware of how we are spending our money. You know, trying not to eat McDonald’s as much, and instead of McDonalds its like, “hey, let’s just make some sandwiches at home.” We are trying to make it so that we don’t have to go out for things as much.
Other interview respondents commented on other skills, such as budgeting, and innovative strategies, such as wisely spending tax refunds, that they are using to build a healthier financial life.
Even though you maintain a budget, I’m just sending more to savings rather than spending it on stuff. I’m just paranoid all the time about having something happen and not having a contingency plan. Savings, savings, savings. We try not to ever take any money out of there.
I can’t stop telling people how important savings is.
Probably paying down the credit card bills has been the best thing, and using that money towards savings so that we can buy a house, hopefully sometime this year.
These findings are confirmed in our survey responses. When asked the question, “As a result of working with STEP, what step(s) have you taken to improve your financial future,” three quarters of respondents report that they are working to pay down or reduce their debts, and over half of these respondents are focusing on maintaining a positive account balance and paying their bills on time.
Both our interviews and our surveys confirm that saving is one of the primary financial goals towards which our former clients are working. The importance of saving was repeated time and again during our interview discussions. When survey respondents were asked about their next step or goal for improving their financial futures, 55 percent of them specifically cited savings in their response.
We are inspired by this finding that everyone in our sample is using some kind of skill to improve their financial life and can clearly identify a financial goal for themselves. This finding confirms three of our six indicators of financial self-sufficiency: that clients will know the next step they need to take to get on the path towards self-sufficiency, that they implement one or more financial self-sufficiency skills, and that they are committed to healthy financial future.
Despite the overall reduction in financial stress, many of the former clients we interviewed report that times are still tough. A few of our respondents talked about regularly fearing and narrowly avoiding financial catastrophe.
We are surviving but it’s been hard. It’s been a bad year. It’s been really hard since my husband retired.
We still have plenty of moments that knock us down. I call them “Oh, sh*t” moments. We have so many of those but we are trying.
You know, we got the help we needed and it got us through a tough time. Now we are facing another tough time, another situation
Still more of the interviews revolved around discussion of new problems and specific obstacles that inhibit clients’ abilities to live the financial lives they want. Issues with creditors, making payments on vehicles, the cost of childcare, and high costs of living were all significant themes.
One of our interview respondents is currently in the process of applying to STEP for a second grant, claiming that life is still very difficult for his family and that it doesn’t take much to knock them into a state of crisis. He said: “You try to do the best you can, but if there’s no money then that’s it. Like, our dog got sick and that totally threw us off. That’s a tough spot to be in.”
Even among our respondents who are doing much better financially, they reference the struggles they had to endure to get to that place and how difficult it was. They spoke of not wanting to “go back” to the time when they felt as if they were at the end of their ropes. While their stories inspire us, they also enlighten us to the fact that many of our former clients are not yet where they want to be and may be in need of both personal success plans as well as available community resources.
Throughout our interviews, we consistently learned about some of the pressing external problems that take their toll on our clients and their families. First, a consistent theme in the interview data was how the cost of living in San Diego (or Southern California more broadly) made life extremely difficult for military and veteran families, who often have little or no choice about living there. Two of our interview respondents went so far as to say that they don’t know whether they would have survived if they had not moved out of state.
Honestly I don’t think we would have survived if we stayed in California. Because everything there is so much more expensive. The cost of living [in Louisiana] is so much better, honestly.
We moved into a military housing community so that has helped a lot. The BAH goes directly to our rent and we don’t have to pay utilities. So we are doing pretty good in terms of paying bills on time, because of how much the military housing has saved us compared to the high costs off Base.
Another theme in the data was the importance of affordable housing and manageable commutes from home to Base for active duty respondents. Respondents noted that getting into Base housing was crucial for maintaining their budgets, reducing debt, and building savings. Yet Base housing is not always available and there are often waiting lists and hierarchies for who gets approved for Base housing. Without Base housing, many respondents face the added cost (in money and time) of having to commute long distances to and from work every day. One respondent talked at length about how she had to commute between her home and two different bases an average of 120 miles per day. Not only did this take up all her time, it exhausted her vehicle and her wallet.
Finally, several of our respondents discussed issues with credit and the difficulty they faced in dealing with auto lenders and credit card companies. One of our clients spoke about how she was “praying every day” that a credit company would refinance their auto loan so that they could afford the monthly payments. Another spoke about how she was relying on credit cards to buy food for her family, and the only way to get out of this debt was to devote her reenlistment bonus to paying it off. These issues with creditors are not surprising to us; military personnel often have complicated credit histories. What is surprising to us is the extent to which issues with credit might be preventing our clients from reaching their long-term financial goals.
While all of these issues are beyond the scope of STEP, we recognize that we are an advocate for the military and veteran community, and that it is our obligation to use our platform to make sure these issues are well known. That means communicating these findings to our Board, to local advisory councils, and to other advocates and policy makers.
These findings are not only insightful to the quality of our impact, they also point out areas where we can make some strategic adjustments to make sure that STEP continues to have a positive impact in the community and to grow this impact as we continue to grow as an organization. These findings also come at a moment when we are about to review and update our client services procedures, and we intend to integrate these findings into that process. Based on what we have learned from this research process, we intend to explore the feasibility making the following adjustments at STEP.
The finding that clients are likely to be grateful for STEP, yet not necessarily credit STEP for their reduced financial stress or the very specific and concrete financial skills that they are using, encourages us to improve the way we are communicating our program to our clients. We want them to recognize that they have learned and grown after working with STEP. We would like to stress the following in our engagement with clients:
We also want to slightly adjust the level of engagement we retain with former clients while still staying committed to our solution-focused brief therapy approach. To do this we intend to make the following adjustments:
We will discuss these recommended changes with our board and staff and use them as guides when revising the client services procedures and refreshing the case managers’ training.
We want to share our findings with relevant parties who can assist us in being advocates for the military and veteran families in our community as well as across the nation. This includes talking to active duty military commands, veterans groups, and our community partners about the issues that continue to affect these families. We intend to do the following to communicate our findings widely:
Participation in the Project Impact research process has clarified our articulation of our impacts and yielded several important insights into the quality of the impact STEP is having in the community we serve. We have not only learned how to design and implement these data collection methods, we also have a plan for how to sustain both qualitative and quantitative data collection into the future. We have learned that while it is difficult to connect with our former clients, once we do they have several positive things to say, including that we offered crucial support during times of crisis and that we reduced their financial stress dramatically. Even if they cannot explicitly credit STEP with current financial wellness levels, we are encouraged that clients are retaining some of the more specific goals that we stress during our solution-focused interventions and that everyone is taking at least some small step towards achieving their financial goals. However, our work is far from over. Many of these families still face dire financial situations, which are often made worse by military and society problems beyond their and our control. By implementing these insights into our program and advocacy efforts, we are optimistic that STEP will continue to have a positive impact in the military and veteran community. We are committed to serving them, because they have committed to serving our country.
1. What were the most useful things you learned through this program?
2. How has this made you think about your life (or your family) differently? (Know-Believe)
3. What have you learned about financial goal setting?
4. How has this changed what you believe is possible for you/your family’s financial future? (Know-Believe)
5. How has your ability to pay bills on time changed as a result of this program?
6. How have you changed your lifestyle to make this possible? (Do-Become)
7. What has been the most exciting part of maintaining your budget?
8. How have those kinds of things made you more committed to your financial goals? (Feel-Love)
9. What has been the most stressful part of maintaining your budget?
10. How do you stay motivated to maintain your budget during those times when you're feeling stressed? (Feel-Love)
11. Before being part of this program, what were your biggest fears when you thought about your finances?
12. How have you cultivated the confidence to stick to your goals in spite of those fears? (Feel-Love)
13. What is a skill you have implemented that you wish you had learned as a child?
14. How does this change the way you interact with your children? (Do-Become)
15. What skills have you been talking about with people close to you?
16. What have they noticed has changed in you as a result of this program? (Do-Become)
17. Over the next 2-3 years, what do you want to accomplish financially?
18. How do you want your life to be different as a result of this program? (Do-Become)
Start your post here...
Posted on 09/13/2016 at 10:17:00 AM