Forum Discussion

Former_Member's avatar
Former_Member
Accelerator III
8 years ago

How to create a learning curve for train and test sets from Carey

I have created a gbm model in caret. I wish to plot a learning curve with performance metric (AUC/ROC in my case) on y axis and 1:n.trees on x axis. I want to get performance values on both train and test set for every tree created by gbm. Which is possible in gbm package (n.trees = 1:n.trees in predict.gbm()), but for some reason I can not reproduce the example in caret.

2 Replies

  • eli_donahue's avatar
    eli_donahue
    Community Manager
    8 years ago
    Hi vinodkumarmvn, I'm not sure I understand your goal. Are you trying to display and/or store data from an external tool/system in Aras Innovator? Eli
    Eli Donahue Aras Labs Software Engineer
  • Former_Member's avatar
    Former_Member
    Accelerator III
    6 years ago

    To simulate the whole train () process (including CV, etc.) with various subsets of the training set (10%, 20% ... 100%). In principle, I would like to evaluate whether your training set size will reduce the large variance. For 2. you need the error rate when changing the mtry hyperparameter, but if I understand what you mean, trainFit will calculate the error rate for different CV-folds (although, I suppose, the last figure is the average for everyone).
    do my homework