Lifecycle of a HUMAN Protocol job
The following steps describe a typical smart bounty lifecycle in the context of HUMAN Protocol.
- 1.Launching of jobs: Anyone who is a Requester can create/launch a smart bounty, otherwise known as a job, using an API or a dashboard with a Job Launcher. Note: When launching jobs in a blockchain network, Requesters can specify the required Job Exchanges in the job description. Examples of Job Exchanges are hCaptcha, used for basic image labeling tasks, CVAT, used for complex tasks such as video annotation, and Audino for annotation of speech and audio.
- 2.Misuse flow (ML specific): If a Worker marks a task within a job as inappropriate, or broken, the task is frozen by the exchange until it can be settled within the misuse flow. This will initially be handled by the HUMAN Protocol Foundation. Going forward, we propose to leverage our network of users by making abuse detection another task on the network, which can avoid a single point of failure and aid the decentralization of the network. In this proposition, the inappropriate image is distributed to several more Workers to verify that the content is inappropriate. If the Workers verify that it is inappropriate/explicit, the network will slash the stake of the Job Launcher. The slashed funds can then be redistributed to the first Worker who saw the content, and those who verified. For now, however, this checking will not be on-chain or decentralized. Essentially it is the Job Launcher which puts up the stake to guarantee that no inappropriate material is launched within a job. In the case that inappropriate material is found, the Job Launcher’s stake will be slashed. The Job Launcher will have the card details of the Requester, and can seek remuneration for the slashed amount from the Requester. Naturally, this flow will encourage Job Launchers to have their own content moderation systems.
- 3.Routing of jobs into blockchain networks: After proper validation, the job reaches the Routing Protocol flow. The RP flow functions like a decentralized switchboard connecting different parts of a job’s lifecycle to the correct paths and elements. HUMAN Protocol’s objective is to ensure that jobs get connected to the appropriate blockchains depending on the nature of the jobs. Every blockchain has various implementations of HUMAN Protocol services, like Fortune, a demonstration app, the oracles, and the smart bounties.
- 4.Accessing jobs and completion of job tasks: Workers can use the HUMAN App to view existing jobs and choose which tasks they would like to complete. They may also use third-party apps such as U App by Blockchain Helix (see grants below) to view and complete tasks.
- 5.Verification of Workers’ responses: Once workers submit answers or responses to tasks within a smart bounty, it is critical that the quality of those responses is verified. This is achieved through the Recording Oracle – where applicable and possible – and the Reputation Oracle.
- At the level of the Job Exchange, the Recording Oracle verifies Workers’ response quality on a rolling basis for each job by: (a) Analyzing Workers’ answers/responses to tasks contained in a job and verifying whether all of the individual outputs meet the quality threshold provided by the Requester. If any discrepancies are found, the Recording Oracle makes the job available to other Workers as well and checks their responses to verify whether the quality is similar to the previously recorded responses; and (b) Writing the answers/responses submitted by Workers to the blockchain through the RP flow.
- Once the quality of all responses is verified, the overall output is relayed to the Reputation Oracle. The Reputation Oracle ensures that the overall response quality for a job meets the defined threshold based on:
- A reference dataset that maps various job categories to their characteristics;
- Answer/response quality benchmark supplied by the Requester when launching a job; and
- The holdback validation set, which is an optionally provided sample of labeled data by the Requester.
- The Reputation Oracle then sends the job results to the Requester.
- 6.Payment to Workers: The Reputation Oracle updates the Workers’ reputation scores, adjusts the Workers’ payouts based on their reputation scores, and executes the payouts. In case of Workers who have not earned a reputation score by completing tasks previously, they are assigned a starting score. Consistent top performers are more likely to be paid higher than consistent poor performers.
Last modified 4mo ago