Disclosed is a memory device and an operation method thereof. The operation method of memory device, comprising: programming a plurality of sub-matrices including at least one of non-zero element of a rearranged matrix to a plurality of operation units of the memory device; and programming a mapping table into a working memory of a memory device. The rearranged matrix is generated by rearrange a plurality of columns and a plurality of rows of an original matrix according to the positions of a plurality of non-zero elements of the original matrix. The mapping table comprises a correspondence of row indexes between the original matrix and the rearranged matrix, a correspondence of column indexes between the original matrix and the rearranged matrix and a correspondence between the sub-matrices including at least one non-zero element and the operation units storing the sub-matrices including at least one non-zero element.