COMBINED VISION AND LANGUAGE LEARNING MODELS FOR AUTOMATED MEDICAL REPORTS GENERATION

    公开(公告)号:US20230386646A1

    公开(公告)日:2023-11-30

    申请号:US18320841

    申请日:2023-05-19

    Inventor: Ajay Tanwani

    Abstract: A method of generating a medical report is presented herein. In some embodiments, the method includes receiving a medical image and at least one natural language medical question, extracting at least one image feature from the image; extracting at least one text feature from the question; and fusing the at least one image feature with the at least one text feature to form a combined feature. Some embodiments further include encoding, by an encoder, the combined feature to form a transformed combined feature; computing a set of prior context features based on a similarity between the transformed combined feature and each of a set of transformed text features derived from a set of training natural language answers; and generating, by a decoder, a first natural language answer conditioned on the transformed combined feature and the set of prior context features.

Patent Agency Ranking