Salford Systems CART - the trade off between Purity and Accuracy -- and its application to The Kingdom

Something I learnt during Salford Systems CART training.

High purity à low accuracy

Low purity à high accuracy

As in, if we want to minimise the impurity function (I), which CART uses,

·         Under GINI splitting method is: which is also

·         Or Under Entropy Splitting method is:

If we want to minimise the impurity function as in we want to maximise the purity that all the records put into the classes actually belong to that class, e.g. we want the blue category as below screenshot to have as less reds as possible (hence pure), then we have to make the blue category more difficult to enter by reducing the prior probability for blue category to 0.1 and increase the probability/ease to class into red category to 0.9 (they have to add up to 1). However, what you get is that at the cut off line, you lose quite a bit of blues that are to the left of the cut off, so the accuracy of the splitting decreases.

However, if you want to capture all the blue records (hence increase splitting accuracy) then you want to shift the cut off line to the left, and you can do so by changing prior probability to (red-0.1 , blue 0.9). Hence increase the ease that any record can be classed into the blue category. But by doing so, at the same time you get more contamination of reds in the blue category. Hence the purity of the class decreases.

That is the trade-off between accuracy and purity. It is impossible to be both 100% accurate and 100% pure at the same time because they are inverse functions of each other.

And I heard God say, “Don’t you know that my church is also like that?”

High purity in Holiness à low accuracy – low number of people streamed into the Kingdom

Low purity in Holiness à high accuracy – high number of people streamed into the non-Kingdom that is hell.

And in order to establish a kingdom whose citizens can last forever, it is clear which path God chose in the trade-off between Purity and Accuracy…
But He did give a way out in His higher purpose that surpasses the records He streams into one class or another à He gave them (us) salvage-tion (salvation). So our analogy, let blue equate to the kingdom’s class/category. The amazingly amusing thing is that – with His blood He can wash us so pure, that He actually turns the red records into blue records – for those who do come to Him. For such people who come to Him, is the kingdom of heaven prepared for them. Confess your sins, and produce fruit in repentance, and He will wash you white as snow.

So that even with the same prior probability – i.e. the same high standards in the split, you WILL get more accuracy! And this is done, but Him saying through His dead and resurrection through the cross – “I have got enough righteousness for Me, and I have got enough for you – too.” When His blood covers you, and His righteousness caringly shared with you, your colour flips from red (under God’s wrath) to blue (colour of royalty in The Kingdom).

And there will be no more condemnation to those who are in Christ. (Romans 8).

Oh, Hallelujah! What a relief.

CART screenshot From: http://www.salford-systems.com/en/training/videos#part-5