Systems and methods that use a neural network architecture for extracting interpretable relationships among predictive input variables. This leads to neural network models that are interpretable and explainable. More importantly, these systems and methods lead to discovering new interpretable variables that are functions of predictive input variables, which in turn can be extracted as new features and utilized in other types of interpretable models, like scorecards (fraud score, etc.), but with higher predictive power than conventional systems and methods.