In a system for improving performance of tensor-based computations and for minimizing the associated memory usage, computations associated with different non-zero tensor values are performed while exploiting an overlap between the respective index tuples of those non-zero values. While performing computations associated with a selected mode, when an index corresponding to a particular mode in a current index tuple is the same as the corresponding index from another, previously processed index tuple, the value already stored in a buffer corresponding to that particular mode is reused either wholly or in part, minimizing the processor usage and improving performance. Certain matrix operations may be iterated more than once so as to avoid the need to store a large partial result obtained from those operations. The performance overhead of the repeated operations is not significant, but the reduction in memory usage is.