Five years of intense work and partnership with internal data scientists to build machine learning (ML) models is now paying off for the treasury team at one mega-cap corporation that has significantly improved the accuracy of cash flow forecasts used to hedge exposure to 14 foreign currencies.
- The team’s senior manager for FX and commodity risk recently provided an update on the multiyear project to members of NeuGroup for Foreign Exchange at the group’s second half meeting sponsored by Morgan Stanley.
Accuracy and time savings. The member described the legacy cash forecasting process that involved about 180 forecasters across the globe submitting every month as “extremely painful and inaccurate.” Five years later, with ML models, “we’ve seen about 25% improvement in accuracy,” she said in response to a question from peer group leader Andrew Weber. - For cash forecasts of 12 to 18 months, often the most accurate for the company, the average variance to actuals using the ML models is less than 10% across all 14 hedged currencies and as low as 5% for some, a vast improvement that impressed other members.
- Treasury’s cash forecasts for shorter periods, like one month, may not be as accurate, the senior manager said. But by that time, the company has done “a lot of the hedging already.”
- Time savings are another major benefit of overhauling the forecasting process. “When we calculated, it was about 3,000 hours a year,” the member said. “It really stemmed from the amount of time the 180 forecasters spent to submit,” she explained.
- “And it’s also the time my team used to spend chasing people who didn’t submit or for reaching out to people who submitted something that’s just completely crazy-looking.”
The testing process. With almost all treasury transitions to new processes, running the old and new methods in parallel not only provides comparative data on accuracy or effectiveness but can help smooth the internal road to change. In this case, the team ran the legacy and ML forecasts in parallel for about seven months. Then the team switched to “reverse parallel” period of three months, with the ML model forecasts “live” and the legacy forecasts a backup.
- “I think that helped leadership get comfortable because we had quite a few months of results before we ever actually pulled the trigger to tell the forecasters to stop submitting,” the member said. “As of this month, September, we will no longer be doing reverse parallel.”
A huge lift for treasury. Among the reasons it took years to arrive at that point was the need to build custom machine learning models for each of the 14 currencies the company hedges (it has exposure to 25). Managing the tagging process and the bank statements the models are trained on required the member’s team to work with finance teams abroad and the data scientists in the U.S.—a huge lift.
- “We had to help the data science team categorize all of these bank statements they go through and say, ok, what’s payables, what’s receivables, based on the verbiage that comes through on the bank statements. And a lot of these are in foreign languages, so we had to work with the local teams a lot to help us understand what are the flows? How do we categorize these?”
- She added, “It’s been successful, but it took a lot of trial and error to get the tagging process right—and a lot of times with the treasury team helping the data science team understand these transactions on the bank statements, because they don’t understand hedging or cash flow exposures. They’re building the models for us and, as the model learns, we were able to kind of fine-tune parameters for each of the currencies.”
Ongoing maintenance and partnership. Machine learning forecast models depend on historical data (in this case five years of bank statements) but need humans to tweak them when circumstances change. “We have a one-time only adjustment process that we can do if we have a transaction or we’re aware of something that could be coming up that we need to help the model understand,” the member said.
- “We have the ability to input that on top of it because obviously if we have an M&A transaction coming out, the model’s not going to know that. So we have a way to adjust that into the model. And then we have a way to take that back out, so next year it doesn’t predict another transaction.”
- She described other adjustments. “If we see tariffs, and sales in Europe are going to fall off, we work with the data science team; they’re able to tweak, by looking only at more recent months as opposed to the full five years of history.
- “So as we have new items come on, we spend time every single month just working on the model to make sure we don’t have slippage, and then the local teams also do a quality check on that.”
- Treasury meets with the data science team every week, the member said. “So it’s really been a partnership and very labor intensive for both of us. And we’re also committed to the maintenance of the model. So it’s not like we’ve gone live, we’re done, we can just walk away. There’s the maintenance aspect to it. And we’re committed to keep going.”
