Performing customized data compaction for efficient parallel data processing amongst a set of computing resources

    公开(公告)号:US11061734B2

    公开(公告)日:2021-07-13

    申请号:US16264399

    申请日:2019-01-31

    Abstract: Described is a system and method for compacting data into customized (e.g. optimal) file sizes for processing by computing resources. The mechanism may leverage various computing resources such as a cluster computing frameworks combined with a stream processing platform to efficiently process the activity data. For example, activity data of an organization may be processed by a set of jobs (or sub-jobs) as part of a data stream by a set of distributed computing resources. In order to efficiently process such data, the mechanism may compact the data into customized (e.g. optimal) file sizes. For example, the customized file sizes may provide an optimal (or near optimal) amount of data to be processed by each job, for example, to improve performance.

    MONITORING AND VERIFICATION SYSTEM FOR END-TO-END DISTRIBUTION OF MESSAGES

    公开(公告)号:US20200311637A1

    公开(公告)日:2020-10-01

    申请号:US16366922

    申请日:2019-03-27

    Abstract: Described is a method and system for verifying the end-to-end distribution of messages within an on-demand services platform. To provide the ability to monitor and trace such messages, the system may include a specialized and queryable datastore (or database) that allows the system to track the distribution and acknowledgement of messages throughout the services platform. Accordingly, a specialized verification process may be initiated to query the database to verify the receipt of a message has been acknowledged by various components within the system. In addition, the verification process may automatically retry the distribution the messages to ensure particular downstream components have received the message. Accordingly, the system alleviates the need to manual re-crawl various message source to ensure the end-to-end distribution of such messages.

    Monitoring and verification system for end-to-end distribution of messages

    公开(公告)号:US11488082B2

    公开(公告)日:2022-11-01

    申请号:US16366922

    申请日:2019-03-27

    Abstract: Described is a method and system for verifying the end-to-end distribution of messages within an on-demand services platform. To provide the ability to monitor and trace such messages, the system may include a specialized and queryable datastore (or database) that allows the system to track the distribution and acknowledgement of messages throughout the services platform. Accordingly, a specialized verification process may be initiated to query the database to verify the receipt of a message has been acknowledged by various components within the system. In addition, the verification process may automatically retry the distribution the messages to ensure particular downstream components have received the message. Accordingly, the system alleviates the need to manual re-crawl various message source to ensure the end-to-end distribution of such messages.

    PERFORMING CUSTOMIZED DATA COMPACTION FOR EFFICIENT PARALLEL DATA PROCESSING AMONGST A SET OF COMPUTING RESOURCES

    公开(公告)号:US20200250007A1

    公开(公告)日:2020-08-06

    申请号:US16264399

    申请日:2019-01-31

    Abstract: Described is a system and method for compacting data into customized (e.g. optimal) file sizes for processing by computing resources. The mechanism may leverage various computing resources such as a cluster computing frameworks combined with a stream processing platform to efficiently process the activity data. For example, activity data of an organization may be processed by a set of jobs (or sub-jobs) as part of a data stream by a set of distributed computing resources. In order to efficiently process such data, the mechanism may compact the data into customized (e.g. optimal) file sizes. For example, the customized file sizes may provide an optimal (or near optimal) amount of data to be processed by each job, for example, to improve performance.

Patent Agency Ranking