Experimental Design
ADOPEM is a savings and credit bank based in Santo Domingo, Dominican Republic serving primarily low-income, urban individuals and small businesses throughout the country. In addition to extending loans, ADOPEM offers savings, insurance, and remittance products. It also operates a training center, with programs including basic computing, entrepreneurship, and specific trade skills. In the year before this experiment was launched, ADOPEM was actively planning to launch a dedicated financial education program and was interested in evaluating different approaches.
Researchers worked with ADOPEM and Dominican training experts to develop two alternative financial education-training programs. The standard accounting treatment offered a traditional, principles-based course in basic accounting techniques. Topics covered included daily recordkeeping of cash sales and expenses, aggregation of daily records into monthly reports, inventory management, accounts receivable and accounts payable, calculating cash profits, and investment planning. The materials and capacitator training program for the standard accounting treatment were based on the financial education program designed by Freedom from Hunger, a US-based nonprofit organization, together with the Citigroup Foundation and adapted to local conditions. The rule-of-thumb treatment taught participants simple rules for financial decision making, focusing on the need to separate business and personal accounts. In addition to presenting several strategies for physically separating business and personal funds, the rule-of-thumb treatment taught how to estimate business profits by simple changes in business cash on hand, paying oneself a fixed salary, distinguishing business and personal expenses, and easy-to-implement tools for reconciling accounts when business funds have been used for personal expenses or the reverse.
In both treatments, clients received record-keeping books, handouts, and homework assignments to reinforce ideas or techniques from the meetings. Both classes were offered once a week for three hours at a time. The standard accounting treatment lasted for six weeks and the rule-of-thumb treatment for five. The first three classes of both treatments covered consumption, savings, and debt management. The final three classes of the standard accounting treatment comprised basic cash accounting, distinguishing business and personal expenses, calculating profits, and working capital management. Classes four and five of the rule-of-thumb treatment focused on separating business and personal money and estimation techniques for calculating profits.
Attendance for classes one through five did not differ across the two treatments. The sample consisted of 1,193 existing ADOPEM business or personal loan clients from Santo Domingo. Of these, researchers assigned 402 to the accounting treatment, 404 to the rule-of-thumb treatment, and 387 to a control group that received no additional training services. The treatment was assigned at the individual level and administrative data was used to stratify according to loan size, years of borrowing, and whether or not a client maintained a formal savings account with the bank.
Baseline survey data was not available at the time of the stratification. ADOPEM made no additional policy changes concurrent with the training program. The treatment was conducted in two waves. The first wave, comprising 302 treatment assignments, was conducted from March to May 2007, and the second wave, comprising the remainder, ran from July to August of the same year. All courses were taught by qualified local instructors. The majority had university degrees and experience with adult education, in most cases with ADOPEM directly. Courses were offered at seven schools throughout Santo Domingo and scheduled based on preferences elicited during the baseline survey. In addition, the course was heavily subsidized. Fees were randomly assigned at RD$200 (approximately US$6) or zero, relative to an overall program cost of approximately RD$700. In order to begin understanding the potential limitations to classroom-based, financial training, we also randomly assigned half of the people in each of the treatment groups to receive follow-up training consisting of in-person visits of a financial trainer to the micro-entrepreneur’s business. When necessary, the trainers reviewed the class materials with the entrepreneurs, helped clarify any questions they had, and reviewed their account records, if any. The purpose of the on-site visits was to ensure that individuals understood the material and were capable of implementing their newly acquired financial accounting skills in their businesses. This structure helps to differentiate the channel by which training affects the participants.
Researchers constructed the original sample frame based on administrative data collected by ADOPEM in the ordinary course of operations. Beginning in November 2006, researchers conducted a baseline survey of each study participant using a professional survey firm unaffiliated with ADOPEM. They collected information on household and business characteristics, business practices and performance, business skills, training history, and interest in future training. The endline survey was conducted during the summer of 2008, at least 12 months after training was completed. Researchers augmented the surveys with administrative data from ADOPEM.
Anecdotal evidence and discussions with ADOPEM suggest an unusually high level of program dropout, business closure, and out-migration from the Dominican Republic by the sample population in response to Hurricanes Dean and Noel and Tropical Storm Olga, which flooded large parts of the country and caused catastrophic damage. The survey team utilized various forms of contact information from baseline and administrative data as well as credit officers in the field in its efforts to locate all individuals in the study for the endline survey. Researchers collected endline data for 87 percent of participants reporting at baseline.
Self-reporting bias raises concerns about any measures of business management practices. To allay such concerns, researchers construct an objective index of financial reporting errors. They classify as an error any report of bad period sales greater than average or good; average period sales better than good; or average period profits better than good period sales for each of daily, weekly, and monthly reported outcomes. Along the same lines, researchers compare self-reported profits to profits calculated from respondents’ own revenue and expense detail. The differences are large, which researchers is due to the fact that respondents fail to remember and, hence, underreport their various detailed business expenses. Thus, researchers are cautious when interpreting any profit measure as a stand-alone outcome; however, they expect that if either treatment improves financial controls, the difference between the two profit measures would become smaller.
Researchers run an OLS regression for non-binary outcomes, where the endline value of the outcome variable of interest is a dependent variable. The regression equation includes an indicator for being assigned to the accounting treatment; an indicator for being assigned to the rule-of-thumb treatment; baseline-measured covariates including business types, loan size, and participation in an ADOPEM savings account; and the pretreatment measure of the outcome variable. For binary outcome variables, researchers estimate a linear probability model following the same specification. They restrict the sample to only those individuals who report owning a business in the endline, so answers to all business outcome and performance measures (e.g., weekly revenues or keeping business and personal accounts separate) are well defined. Standard errors are clustered at the barrio level to account for community-level shocks to business conditions.
Researchers test for heterogeneous treatment effects with respect to skill level, prior interest in training, and baseline quartile of business financial practices by re-estimating the regression equation, while restricting the sample in turn to each of the partitioning subgroups. They construct index measures for three families of outcomes: business practices, personal financial practices, and business revenues. Within each category, they rescale each outcome, such that larger values indicate better values for the individual or business, and convert each measure to a z-score. For each category, researchers then construct a summary measure and test whether the training treatments affected the set of outcomes within the category.
The businesses in this study tend to serve spatially local markets. Thus, researchers assess potential spatial externalities from the training. These spillovers can take two forms. First, there may be positive knowledge spillovers, with trained clients actively passing on newly acquired knowledge to their peers or neighboring businesses observing and mimicking improved management practices. Second, business outcomes for the treated may improve either by expanding the overall market, “growing the pie”, or by a reallocation of revenues from control firms to the treated, “business stealing”. Researchers construct a measure of the distance between every pair of businesses and density measures for the total number of firms located within 0.5 kilometers of business, as well as the number of firms in the accounting and rule-of-thumb treatments. They also characterize the proximate firms by whether or not they operate in the same basic industry. The regression equations use as outcome variables both revenues and business practices.