Options
It is easy to fall into a cargo cult approach to proof of stake, adding it anywhere and everywhere simply to reduce token velocity. This tends to do a disservice to adoption by increasing friction, and often does not fit the real world dynamics of a protocol.
In our case, both the fundamental design and many different components work separately and together to prevent abuse and ensure it is not rewarded. However, as adoption grows it becomes more attractive to supplement confidence scores with other mechanisms.
Rather than risking staked tokens, we propose instead to treat the current HMT balance of the ETH addr requesting tasks to complete as a priority input when choosing which jobs and tasks to serve from the Exchange open order book. Holding more tokens will increase the probability of higher earnings due to better access to the highest bid jobs available on the exchange.
Similarly, we propose to sort equal bids from Requesters within the order book on the basis of HMT held by the requester address: holding more HMT gives higher execution priority to jobs launched by that account. In this protocol’s implementation, primary operation of the fairness algorithm that balances the increase of value received by existing vs. new network participants is devolved to individual Exchanges. We plan to release multi-agent simulation results to help understand this new model and its effects in various scenarios.
One area of research now finding applications in machine learning and other fields is factored cognition: decomposingmorecomplicatedworkintoitssimplestcognitivecomponents.Practical applications of this idea map very nicely onto the HUMAN Protocol.
For example, an Exchange may publish a custom high-level job type like “extract structured data from each page of a scanned form and return it in the schema I provide.” This job has many sub-tasks: identifying text fields in the image, matching their type to the provided schema, reading the contents of the field, and so on.
Each of these sub-tasks can be mapped onto a standard job type provided in the initial Exchange reference implementation. That work can then be distributed across the entire network of Exchanges publishing those job types (based on real-time cost and reputation), and the sub-results combined to provide a final result. The exchange fee mechanism has been designed to incentivize creation of these higher-level value-added job types, allowing Exchanges to set, publish, and vary their own fees based on market conditions.
Many larger tasks can be decomposed into sub-tasks requiring very different levels of trust. Consider our document scan example above. Seeing the entire page requires high trust if it contains personally identifiable information (PII). However, once a field has been labeled by type then reading its contents may require no trust at all. A social security number has no value in isolation: it becomes PII only when combined with a name and address.
We can use this understanding to construct Exchange hierarchies that run the (often small) task portions that require trust into an appropriate venue, like a private Exchange with registered members. The results of that labor can then be used to generate trustless tasks to load into high volume and lower cost public Exchanges.
A common desire in many forms of assigned labor is to limit the audience within a labor pool in some way. For example, a particular job may be intended only for people who are strong English speakers, people who live in the United Kingdom, or people able to pass a test containing similar questions.
Restricted Audiences are the HUMAN Protocol’s solution to this requirement.
- 1.Requester defines a r estricted_audience property in the job manifest. For example, “restricted audience”: { “lang”: [ { “en-us ”: { “score”: 0.9 } } ] }
- 2.Exchange restricts served tasks from this job to clients whose browser reports en-us at 0.9 or above.
- 1.Requester specifies test in r estricted_audience property of job manifest and required pass rate. In manifest “restricted audience”: { “tests”: [ { “uuid”: “uuid-goeshere”, “uri”:”https://test.com/1.json”, “score”: 0.9 } ] } In test JSON List of tasks and questions, following standard manifest taskdata format. (See reference implementation in hmt-contracts repo.)
- 2.Exchange restricts served tasks from this job to clients who have passed the test at the specified score. Manifest is encrypted, keeping test data private.
As demonstrated on hCaptcha.com: websites with pre-existing ETH accounts don’t need to enroll anywhere to get started. They can simply drop in the JavaScript embed line and set their ETH key on the server side. However, reputation oracles may choose to require registration on a whitelist prior to transfers in order to meet country-specific compliance restrictions.
Websites without ETH accounts can also get started almost immediately: after registering with an exchange they receive a temporary key that is in fact an Ethereum account managed by the Payment Agent (tied to their email address) and may enter their own ETH account at a later date.
Allow “online” (streaming rather than batch) delivery to requester-controlled results endpoint with immediate payment. No reputation oracle analysis and no validation set required. Requires MVP attack mitigations described in Attacks.
In some scenarios it is valuable to receive low latency human input on model inference quality to e.g. guide online learning or metalearning processes. When this behavior is required the requester can create a smart bounty with the job JSON specifying the “online” job type and a receiving server, prepaying tokens to the smart bounty. The nominated exchange provides direct REST API access for datapoint entry. All job datapoints submitted via REST API run at the highest bid price in order to guarantee queue priority.
Results have low individual value and high aggregate value for this service. While all inter-component communication and final results are encrypted, it adds only minimal value to use e.g. homomorphic encryption on that data for intra-job storage. Not required for MVP, based on customer discussions.
Exchanges and oracles need to monitor the blockchain for revocation notices. We currently have an abort function in the smart bounty, callable only by the requester, that terminates a job so long as no payouts have yet been made. In order to prevent abuse, a duration and job size-dependent cancellation fee is charged to the requester, paid to the exchange and oracles.
Several entities in the system need to both sign/ encrypt arbitrary data (of which a portion may be controlled by an attacker) and make contract calls to transfer funds.
Best practice is to rotate signing keys and avoid key reuse between arbitrary data signing/encryption and ETH transfers. We have designed and built a smart contract solution that may be interesting for the wider Ethereum community: an authenticated blockchain key-value store that allows arbitrary metadata to be uniquely and verifiably tied to an ETH addr. (Please see the hmt-ethkvstore repo for more details.)
This allows us to provide a lookup layer between the ETH address of the agent or oracle (doesn’t change) and the signing key or other metadata (may change).
This means that we start with agent security guarantees that are only as good as the underlying ETH addr key management, but additional safeguards and layers of defense are possible. For example, the public signing key can be published via multiple avenues and automatically checked, and any unexpected disagreement can alert the recipient parties to engage manual intervention.
We may not want to allow the requester to choose the validation set. Why? They might choose non-representative questions, when a random sample would work better. The reputation agent can provide a list of data to validate to the requester, or alternately send it for labeling at high confidence and simply let the requester decide whether those answers are good enough to proceed with job submission.
Reputation scores enable early detection of abuse, robust anonymous proofs of humanity, and other functions. MVP: the Recording Oracle keeps track of per client and per-server reputation scores on the fly in order to assist in initial confidence computations. These are passed back to the Exchange along with pass determination, and final computations are made by the Reputation Oracle.
Initial Application: Early Abuse Detection
The Job Exchange can maintain a rolling cache of reputation scores received from the Recording Oracle in order to to aid in real-time abuse identification. At the end of a job the Reputation Oracle then computes authoritative client and server reputation scores on a per job basis, using validation set data (if available) and the Recording Oracle’s intermediate results.
Other Applications: Anonymous Proofs of Humanity
Once a robust reputation score and confidence value can be computed on a per-client basis there are many applications. One example is proofs of humanity. View & click fraud is rampant in the online ad market. Altering advertisers’ payouts based on sampling of client and server reputation scores effectively creates a mechanism to punish sites serving a higher percentage of paid impressions to non-human bots, increasing ROI on ad spend as more humans and fewer bots receive paid impressions.
Reward Mechanism
MVP: Individual users of public exchanges (e.g. hCaptcha.com) already receive an effective benefit in that their user experience improves alongside increased reputation. Users of private exchanges can equally be rewarded by supplementing proof of balance computations in open order book sorting, job execution sequencing, and other mechanisms.
Last modified 4mo ago