Question

When performing post-deployment evaluation of a model, what criteria would you use to assess if the model is still useful? What factors will be present in that criteria?

Answer 1

`Hey,

Note: If you have any queries related the answer please do comment. I would be very happy to resolve all your queries.

We can tell by using the process is called model testing. It’s a set of procedures that pits competing models against each other post-deployment to determine which of them does the best job providing outputs that are not only statistically accurate, but are also easy to translate into metrics that help stakeholders meet key performance indicators, or KPIs. In other words, model testing is all about selecting a model that strikes the right balance between mathematical accuracy and practical usefulness.

A/B testing and multi-armed bandit tests are the best approaches a data scientist can use to ensure a model will be successful across these two criteria post-deployment. Both of these methods are considered tried-and-true in the realms of digital marketing and website optimization, for example, but they stand to be much more widely adopted in data modeling.

Kindly revert for any queries

Thanks.