Examples "too good to be true"?

Question

Hi,

From ML-Lab 6 I got:

and

"My" ROC-Curve does not hug the upper left - and it is a pretty bad model. But nevertheless...I do not think it at all realistic to have extreme "good" examples like here: https://learn.microsoft.com/en-us/azure/machine-learning/how-to-understand-automated-ml?view=azureml-api-2#Confusion%20matrix

Can you suggest good books which teach you what you "really" get from experiments? Not Marketing-Whitepapers of a company showing off how well they did, but the real struggle?

I do not mean "how to do ML", but "real experiments - and failures - and why".

Bye

Michael

Answer 1

@Michael Schmidt You could try running the samples from azureml-examples repo.

There are many scenarios and notebooks with sample data that could give you good understanding of running ML jobs with azure with sample data. If you have your own sample datasets, you can replace them in the samples or modify the dataset and re-run the notebooks to understand the metrics and how they deviate if you change the data.

For example, this notebook is using data from its data folder and if you update the parquet file in test or train with your own data you should be able to check the deviation.

The notebook as is provides good explanation of how to analyze the graphs and what could be done to get better results.

I hope this helps!! Thanks!!

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.