How to Estimate ROI and Costs for Machine Learning and Data Science Projects?

‍What is ROI?

Machine learning and data science projects are an investment to each person involved and comes with associated risks. ROI stands for ‘return on investment’ and is used as a performance metric, usually expressed as a ratio or percentage. In its most basic formula, 

ROI =      Net gains

              Investment cost

Though this formula seems simple enough, in reality these values have to be estimated as closely as possible. This involves a lot of thinking ahead, and many factors have to be taken into account.

Factors to consider in estimating ROI in AI projects

1. Duration

The estimated duration for the entire period, as well as the time period until the project reaches profitability must be considered. This can help in comparing two contemporaneous projects with similar ROI, or even comparing your current project with a similar one in the past.

2. Taxes and fees

Depending on the jurisdiction the project takes place in, the organisation may be subject to different taxes and fees on profits, employees, resources etc. This will affect the investment costs.

3. Inflation

Inflation affects the actual value for money of products and services from year to year. For projects that span years, or even decades, the forecasted inflation can be taken into account.

4. Opportunity cost

For every project you invest in, there will be others that will not be invested in. What would that money have been used for if it hadn’t been invested in  your ML or data science project?

5. Transaction cost

These are ‘expenses’ incurred in the process of developing your project. It may be fiscal expenses, or expenses in terms of employee-hours, power consumption, infrastructure use and maintenance.

6. Profit

It is important to decide early what will qualify as ‘gains’ from this project. It could be direct income or more indirect value to you or your company. If it can be quantified, it can be counted as profit.

Estimating ROI of your Machine Learning project

Now that you’ve considered these important factors, it’s time to actually calculate your ROI on your important machine learning or data science project, decide if it’s worth the investment and change or adjust your plans accordingly

Step 1: Estimate Profit

We’ve already considered the profit above, but it’s time to have a more in depth analysis. Revenue can increase as a by-product of your main project- increase in productivity or development of a profitable product other than what was intended. All these can be factored into the estimated project

Step 2: Calculate Prediction Accuracy

This step is somewhat related to the first:  the precision of your ML model is directly linked to the gains you can expect from your project. How accurate is your algorithm? Data scientists use the term ‘classification accuracy’ as a metric to evaluate the accuracy of an algorithm’s predictions. Mathematically, it is the number of correct predictions divided by the total number of samples inputted, usually expressed as a ratio.

Step 3: Consider Alternative Solutions

The age of artificial intelligence has created a boom in machine learning products, meaning there are a lot of options to choose from. The cost of each one will be difficult to predict with 100% accuracy. This is all the more reason why more than one  product must be considered in a truly unbiased way before the project starts. Alternatives include open source software, outsourcing to competent machine learning solutions teams or even training trusted in house teams.

Step 4: Project Elaboration

How feasible is your project? This is where your project needs to be input from trusted and competent data scientists. During this phase, which can last as long as a month, the exact nature of your project should be refined and documented, the quality of data assessed and achievable level of accuracy determined. This is done through basic modelling. The risks involved and possible mitigations should also be discussed, as well as any costs involved in any corrections that need to be made at this stage. At the end of all this a comprehensive feasibility report can be prepared.

Step 5: Further Research

Much research has to be done in the step above, however you can go a step further and do more research to glean more information, test algorithms, prepare data and do more evaluation.

When all stakeholders are satisfied with the research outcome- a strong, stable algorithm that is uniquely tailored to power your intended project, development can officially start. Congratulations, you have successfully estimated ROI for your machine learning or data science project, and now you  can launch for the stars!