-
Notifications
You must be signed in to change notification settings - Fork 98
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
When should did parameters be dropped? #337
Comments
To add further detail to this issue, calling to an ion based sidetree node with the following request
Results in
When I would have assumed the following result being sufficient
|
The group feels that better language around this topic should be added to the spec. |
Per the current spec, the above |
I agree, we should provide an example (not ION, but similar structure) and a note. |
Needs PR to add explicit warning about how DID params are dropped from the did document and may be present in resolver meta data.... needs PR. |
Assign me. |
Still need to generate PR for this, sorry for the delay :/ |
If we don't get a PR for this before we enter CR, this issue will be deferred to the next version of the specification. @csuwildcat, you have 30 days to generate a PR for this issue. |
I worry that folks don't realize how this impacts Verifiable Credentials.... this means that when you resolve: "did:example:123?version-id=1#key-0" and you get back a did document with verificationMethod: "did:example:123#key-0" You will need to do the following: for JWS, you can just "find the public key bytes and check the signature.... " for linked data proofs, you will need to apply post resolver middleware that "inserts the version" IF the original signature was over the version, OR you will have to be careful about checking the credential... cc @dlongley I worry that if we are not explicit here, one of the key features of the did spec will be implemented very differently by DID Methods, leading to broken verifiable credentials. |
The second value... that is The signed VC can have a Perhaps there is some algorithm somewhere in some spec that is causing an issue here? |
@msporny its rather that there is no algorithm in ANY spec, that handles this properly... including the VC Data Model (does not elaborate on query parameters) and JWT/JWS (which does not understand dereferencing). Best thing for developers would be a set of "Good" and "Bad" example using query params in the verification method and the vc data model. |
I'd be supportive of this. The right place to do that might be the Linked Data Security specs (which is really unfortunate, because they are not normative in any way). We could put an appendix in the DID Core spec on the topic, it would be a bit weird and out of place, but at least it would be there. We could put it in the implementation guide, but I expect that thing won't be done by REC. I think I'd rather document this in the Linked Data Proofs spec in the processing of the verificationMethod algorithm... but then went to look at the latest spec text: https://w3c-ccg.github.io/ld-proofs/#proof-verification-algorithm ... which is just awful, everything is terrible. |
I think this belongs in did core, since did core is defining dereferencing... we either define how parameters are or are not present in the resulting documents... or we stop trying to define dereferencing... the work needs to be done in did core, or its like saying we support HTTP GET, but don't define the response format. |
The idea was that DID Core would define abstract functions with the inputs and outputs of resolving and dereferencing, and that DID Resolution would define algorithms that implement those abstract functions, answering questions such as:
There is such an early algorithm in DID Resolution (see here), but admittedly this is not in sync with DID Core and needs to be updated. |
@peacekeeper regarding query parameters.... they are NOT allowed in the Are they allowed in the |
@OR13 yes the The spec currently says: The value of the id property for a verification method MUST be a URI. Perhaps we should add the following? The value of the id property for a verification method MUST be a URI and MAY be a DID URL. Or even SHOULD be a DID URL? |
@dlongley @peacekeeper @msporny @dhh1128 @kdenhartog This is one of the most important parts of the whole specification. It's the link between did core and all assertion formats that use identifiers, including JWT, VCs, ZKPs.... Its relevant too HTTP Signatures, DIDComm and many other areas.... Whatever rules we have here will shape how useful DIDs are in those other areas. If we are saying that a I think there are a number of assumptions developers will have, which I am not sure this WG as fully considered: Dereferencing from a However, this is NOT ALLOWED per the spec today when combined with relative refs: https://w3c.github.io/did-core/#relative-did-urls didDocument.id cannot include the query params, so relative refs cannot be combined with query params of any format. This kind of ambiguity is likely to cause serious security / interop issues for anyone using query params. I would go so far as to say, its very dangerous for us to not have some concrete examples of VCs / JWTs issued from DID URLs in the spec, or language in the spec that clearly explains that they are not allowed. |
I'm not feeling confident that I grasp all the implications here. I think the sentence that I most need to understand is this one from Orie:
I THINK I'm totally comfortable with that, but I may be missing something. My own plans have always been that the Or is there some use case for DIDComm where it is important to have DID URLs as the value of an |
If you are using JSON-LD, the identifiers from the credential are expected to match the document returned by the resolver... JSON-LD has a whole section of the spec dedicated to handling "documentLoaders" of which DID resolvers are related... for JWTs or other ZKP formats, there is not always an assumption of "documentLoaders" and getting to "public key bytes / verification method details" may not be formally defined anywhere.... for example, OIDC defines this process for JWT Essentially the DID Spec needs to comment on the following:
vs
vs
The reason being that the following URIs are all different, and signing over them / derefrencing them in different ways could lead to interop failures:
If your did document uses relative refs, If you don't use relative refs, you can have the following:
Since The are technically unique... The DID core spec already recommends using
and
Are different URIs.... if Similarly for JOSE, if the
Or vice versa, how does the verifier know which public key bytes to use? |
cc @selfissued @csuwildcat this is also related to "DID / JOSE interoperability"... and the very poor usability of the "VC-JWT" format... since VC-JWT does not have "documentLoader" but also does not define a dereferencing algorithm for Sidetree JWS does not require |
Is this maybe the same problem as the long form / short form thing in Sidetree? Could you do this:
Then dereferencing Or could you have a |
@peacekeeper -- Sure, we can avoid using "client" and "server", but something must be used to label the entities serving similar roles, even if these labels are not generic enough. (They're not HTTP-specific, even if that's often the easiest frame to use as an example.) You didn't provide any alternative terms; do you have suggestions? "DID Resolver" would seem to be the agent that delivers the DID URL, but maybe it's the agent that is resolving the DID to get to the DID URL? "DID Document Requester" would seem to be clearly enough the agent that is dereferencing the DID URL to get the DID Document, but what is the agent that provides the DID Document? (These labels are important, and not easy to settle on. It took months if not years to settle on Issuer, Verifier, Subject, and Holder for Verifiable Credentials -- if those can even be considered to be fully settled, now.) Strictly speaking, HTTP doesn't require DNS, either; |
In my mind it's like this:
Both could be quite simple or quite complex. Both could consist of multiple hard- and software components (even in a classic client/server architecture), but they don't have to. I'd be fine with continuing to use the term "client" to designate the thing that invokes the DID resolution and/or DID URL dereferencing functions. But we agreed at some point that we don't consider the "client" to participate in performing those processes. In other words, in the case of DID URLs the "client" does NOT do dereferencing, that is done entirely by a "DID URL dereferencer" (which can itself be a local process). If we want to discuss this further, we should probabaly raise a separate issue. Two references that could be relevant here:
|
My #544 (comment) may be more relevant here than there... |
I wrote some tests trying to make sense of I am very concerned by the results, most of the working examples are illegal according to did core spec today, and I could not produce a single "working legal example" of The cause of this is assumptions in linked data apis, (like frame) about the relationship between take a look at these examples: https://github.com/OR13/did-params-and-you/tree/master/examples In particular, I believe that @dhh1128 @oed wished to use The reason this is illegal is that Based on these tests, I am not sure how exactly to resolve this issue, but I am leaning towards recommending normative changes to did core, to make the working examples legal. @dlongley @msporny I am concerned that the JSON-LD and non JSON-LD community are not working together wrt normative requirements in did core that impact verifiable credentials.... The did core spec should not be making valid linked data stuff illegal... especially if the non linked data community is going to just ignore the linked data side of the spec anyway... the normative requirements of did core need to work for both sides, and we should not be encouraging many slightly different ways of handling things like PROPOSED CHANGES:
For the best example of something that "works" but is "not legal", see this:
Without the proposed changes, existing credential libraries will be incapable of supporting normatively legal did documents that use query parameters... BTW, while it might seem like this is "only a problem for JSON-LD"... its actually a problem for anyone trying to use did core with verifiable credentials. for VC-JWT there is no defined algorithm for verification method dereferencing (they don't have documentLoaders or frame)... which means that there is even more optionality in terms of how those folks might handle this issue.... if they read the spec today, they will 100% create solutions that work for JWT that are not legal JSON-LD... and then later encounter sever pain as they try and get their did method to "support both camps".... |
Oof, yeah... this is a problem. It's further exacerbated by the fact that a DID Resolver may not be able to change the DID Document that's returned (because it would break a signature over the document, for instance). So, the linked data proof libs might need to be changed so they take an argument for a DID Document id mutator that can be aware of the query param mechanism. If we do that, DID Core wouldn't have to change, the VC libraries would only require one more optional parameter (id mutator), and the rest of the ecosystem can stay the same. That said -- I haven't put in the amount of thought that would be required to feel comfortable with this option -- just throwing it out there as one potential path forward. I expect @dlongley @dmitrizagidulin or @davidlehn will have a better idea of what to do here wrt. software implementations. |
I have more thoughts here but too little time at the moment. My main concern is that people may be trying to do things that they think are secure, but are not; that the trust they are putting in certain proofs just isn't there. We need to more fully explore a clear use case to determine if there's even a solution that makes sense here. By definition, a system where the verifier will blindly accept any past version of a verification method declared by the prover does not support revocation. I said more about this sort of thing in the thread about revocation here: So, I am not a supporter of "just" referencing version-locked verification methods in LD proofs and making it such that verifiers that already accept proofs with non-version-locked verification methods would start accepting these without changes to their software. That introduces a security hole. I would rather have VC proof verification code fail closed in these circumstances and require something "extra" to accept a proof with a version-locked verification method. For example, if you want to enable proofs from the past to be accepted, include an additional "proof of existence" proof on the document and have it reference the version of the verification method that is acceptable. Then, make your resolver return that version when asking for the verification method: {
"id": "urn:mycred",
"...",
"proof": [{
// details TBD, but proves the VC existed at the same time
// as a particular DID Doc version
"type": "MyProofOfExistenceBlockChainAnchorProofThing",
"blink": "blink:btc:23498u2342/f234f23u23f/whatever",
"anchoredResource": {
"id": "urn:mycred",
"didVersion": "did:example?version-id=123124",
"contentHash": "z4917y912374y9743234"
}
}, {
"type": "Ed25519Signature2018",
"...",
"verificationMethod": "did:example#key123"
}]
} In the above, a verifier configured with default software would reject the second proof because the verification method had been revoked. However, it could configure its software to first check the proof existence proof, and if they are happy with it (based on business rules), they could then configure their DID resolver to return the DID Document at the older version when checking the second proof. Then the second proof would verify and all would be well -- and it could be trusted because, presumably, the proof of existence had an acceptable root of trust that was not the same as the prover.
I may be misunderstanding you, but your example and this text contradict the spec. Currently the So examples that use query parameters in the |
thats correct, and but just so everyone is clear... there are currently no normatively legal examples of VCs (or really ANY resources, including the kind that @talltree and @brentzundel wanted supported with my examples say "illegal but working"... meaning a developer will have to violate the spec to do what they want to do... I don't think we can stop them at this point. in other words... most of the use cases for The safest thing to do would be to remove We can't make We need normative language that treats DID URLs as IRIs and lets developers rely on what an IRI is regardless of their chosen representation... we currently have JSON-LD making a distinction between DIDs and DID URLs... where other representations are treating both DIDs and DID URLs as IRIs.... I would be shocked to not see the normatively illegal, but working JSON-LD examples reflected in CBOR and JSON... I don't think the proponents for 3+ representations considered the cost of the ambiguity they created... this is chickens coming home to roost.... ambiguity at this layer of the spec is akin to a terminal illness. |
If we can't reference Orie's statement emphasizes that he's worried about VCs not supporting this feature. I don't care about that, since the credentials I work with don't bind to credential holders with DIDs and therefore don't care about the value of |
I left some suggested alternate wording as a comment on Orie's PR: #548 (comment) |
I don't think this issue is about whether or not |
I have come to think this is generally correct about the DID Document; whatever you would have gotten within that DID Document today, when But I think there must be some difference in the surrounding metadata or other encapsulation — which on that future date should indicate that at that time you dereferenced/resolved a URL that included the In other words, I'm thinking that when you dereference/resolve a DID URL that includes That said, I wonder how you or I or anyone (as the dereferencer/resolver) is to know that they should be requesting I hope that's clearer than I fear... |
I am aligned with @dlongley 's comment. I also agree with @TallTed . Regarding the final question, "how to know to request version-id=X if it's never mentioned anywhere" -- the peer DID method that I wrote would use the hash of the DID doc as the value of X, and the Indy DID method would use the ledger transaction identifier, I think. |
@dhh1128 -- You provide what might be used as values for X in two DID methods, but it seems to me they are just as opaque and undiscoverable (if not more so) as simple version numbers... e.g., how would I know the hash of a DID doc I haven't yet retrieved/requested (but someone else has used or relied on), in order to use that hash sometime in the future? At least with an |
I am not opposed to Regarding the question, "How would I know...", I would say that if another party is using a DID doc, they provide the hash to you so you can reference it. |
Just a quick update to my own earlier comment: |
I think I have managed to cut this issue free from the other PR on "rotation/recovery", in changes to that PR, I opted to reimplement the PR because it was faster than slogging through rebase.... #569 But I believe I have addressed the spirit of all proposed changed in that PR ^ Regarding this issue I want to make a couple of points.
I know Indy Credentials don't follow the VC Spec regarding If Indy Credential Schemas are just plain old json, this is not an issue.... and I would suggest requesting them with However, if you want Indy Credential Schemas to be both I believe that all the "illegal but working" stuff would apply to HyperLedger Indy Credentials and their Schemas. I have never seen an Indy Credential Schema as JSON DID Document, despite @brentzundel @talltree @jandrieu arguing over Perhaps it is time to provide an example of Indy Credential Schema that is represented in JSON and stored on a ledger, and then we can answer the question of wether did core supports what you are wanting to do more concretely. To be clear.... I want Indy Credential Schemas stored on ledgers to be valid JSON and JSON-LD and be compliant with DID Core.... In absence of evidence that they are, I am assuming they are not... How HyperLedger Indy Credentials ended up being handled in the VC spec is one of its biggest failures, we should try and hold ourselves to a higher standard. I would like to prove that what Indy Credentials are doing is legal wrt did core, and I would like to include Indy Credential examples in did core, like we have for JWT and LD Proofs already: https://w3c.github.io/did-core/#proving I don't have the indy credential skills to actually do this... we should find someone who does, and get them to add a PR before DID Core closes the book on Indy's failure to define stuff properly, the same way the VC Data Model did. What we need:
Ideally these examples would use |
I consider this an incorrect characterization. Please read section 4.8 of the VC spec carefully (where the proof section is formally and normatively defined) and section 7.4, where it is discussed again. I believe you will see that What is really happening in this case is that Indy doesn't follow the assumptions/tribal knowledge of JSON-LD advocates (or it doesn't follow the JSON-LD spec), NOT that it doesn't follow the VC data model. (It is true that Indy credentials don't follow the VC spec completely, but
Yes, today they are just plain old JSON. Indy schemas are not retrieved from a ledger using HTTP content negotiation, so their MIME type is irrelevant right now. But this is a good suggestion to keep in mind for the future.
An Indy Credential Schema looks like this (as documented in https://github.com/hyperledger/indy-sdk/tree/master/docs/how-tos/save-schema-and-cred-def, published 2 years ago): {
"id": "1",
"name": "gvt",
"version": "1.0",
"attrNames": ["age", "sex", "height", "name"]
} The Indy schemas tomorrow may be far more elaborate; Brent has published a series of detailed Indy HIPEs and Aries RFCs about their format.
I am mildly annoyed with the awkward way that Indy credentials map to and from the VC spec, but deeply frustrated by the human communication around the topic, which has distorted the problem in size and nature. I don't believe the latter problem will be fixed by spec content, though improving the spec content is a noble exercise.
The issue I have with this is that I think you're imagining a false parallel. In their issued form, Indy credentials are not intended to be shared. Rather, a new JIT credential is created from the issued artifact every time sharing occurs. Therefore, looking for issued Indy credentials to map onto a spec about what should be interoperable during a sharing operation can have little benefit. What would be a useful exercise is to show how an Indy proof (the JIT derived VC that's shared and is thus an interoperability surface) maps to the VC spec. I raised a PR against Indy SDK in August 2020 to do that. It has not been accepted because it requires a dirty hack to fetch private variables from an internal cryptographic data structure. Rewriting it to be a pure JSON transform (no dirty hack) is probably a couple weeks of effort that I haven't had time to do. But regardless of the PR's state, here is the documentation associated with it: https://docs.google.com/document/d/1ntLZGMah8iJ_TWQdbrNNW9OVwPbIWkkCMiid7Be1PrA/edit#
As I sat at RWOT 9 in Prague (Sep 2019), clapping for the announcement that the VC spec was at long last finalized, the "how to write an Indy schema to the ledger" doc that I cited above was already almost a year old. It had been carefully tested, and it embodied techniques used in shipping code in production. The conclusion I draw from this is that characterizing what happened with the VC data model as a failure on Indy's part to define stuff properly is really unjust. Long before either of these events, Lovesh and I submitted an enormously detailed PR to the VC spec, trying to help the spec broaden its understanding of the JIT derived credential phenomenon at the heart of our ZKP approach. Getting the PR to affect the VC spec proved to be a slog; certainly we invested hundreds or even thousands of person-hours. Brent eventually took over that effort for us, and heroically got the effort to some semblance of coherence and closure. We did succeed in making certain features optional rather than required (e.g., an I think the deeper issue here is that we have a fundamental disconnect about the scope or legitimate purpose of these standards. The VC standard is a data model standard. It is not supposed to say how that model is realized, except in very general ways, and it is not supposed to achieve interoperability at anything except the abstract data model level. The fact that it provides some examples does not mean that the examples are normative -- and the spec explicitly makes this point, repeatedly. But the community has used the spec like a data format standard anyway, and I feel like I've been clubbed over the head by critics who refuse to acknowledge that their favorite conventions are not actually as standardized as they claim. Similarly, the DID spec is supposed to describe what is possible and fundamental about DIDs. It is not supposed to describe all the details about how DIDs are used in VCs, or how DIDs are used to publish schemas, except at a very high level. Putting in examples that are non-normative should not be necessary, since we shouldn't put too much stock in those examples anyway. Much of the reason why I have been uncooperative on this topic is that I don't want to reinforce the false impression that these standards actually standardize their non-normative examples.
I will leave this to Brent and Drummond. |
The following indy credential schemas are both valid Assuming that someone in hyperledger indy ecosystem can provide a context for indy credential schemas, hosted at: https://w3id.org/indy/v1 or similar (happy to help set that up, just give me a list of terms and their definitions.). Example 1
Example 2
The following examples are illegal according to did core today, for both Example 3
Example 4
Notice that This is the normative requirement that makes example 3 and 4 illegal: https://w3c.github.io/did-core/#property-summary "id is required, and it must be a DID" A DID cannot contain It is this exact same requirement that makes the verifiable credential examples "illegal, but working": https://github.com/OR13/did-params-and-you/tree/master/examples To be blunt... this normative requirement is likely to get ignored by any developers attempting to store arbitrary data on a verifiable data registry. Interpretation of this requirement might lead to a preference for JSON over JSON-LD see the tests I linked for evidence. @brentzundel and @talltree have already expressed a desire to store arbitrary data on a verifiable data registry. @jandrieu has objected to this, but ironically not objected to the abstract data model and the rest of the spec that essentially encourages this by describing extensibility features of DID Documents and saying that unknown properties must be preserved. I recommend the following to any DID Method implementers:
|
@OR13 please correct me if I don't understand (sincerely), but I think dropping DID Parameters for schemas or other resources stored on a ledger may only be a problem if the schema or other resources are delivered via a DID Document. My understanding of the consensus the group came to (which resulted in |
@brentzundel yes, although i am not sure dereferencing is defined well enough.... I guess my point is that did core makes a number of json based document formats illegal, (by preventing IRIs like |
I'm curious what the current status of this is. And at risk of making things more complex, how would the "hl" query parameter (hashlink) be handled? It's infeasible to include the hash of a document within the document, so that presents an obstacle to treating "hl" as a query param for inclusion in the DID document as described in this thread. |
IMHO, the current status is "unlikely to be resolved any time soon". The DID WG published the v1.0 REC and is expected to now go into maintenance mode. We will gather implementation feedback over the next year or two and then pick this item up in the v2.0 work (unless it's clear that this particular issue is causing significant interoperability problems).
Yep, good point, and I expect it would be reasonable to drop it. Similarly, there could be an argument made for keeping it "the All this to say, there are some fundamental architectural discussions that might need to happen around this specific item and, for the time being, it's probably safer to just strip the Hope that helps. |
Consider:
did:example:123?q1=1&q2=2#fragment
=> did document.should didDocument.id ===
did:example:123?q1=1&q2=2
ordid:example:123
?this is particularly concerning for interactions with the
initial-state
parameter used by sidetree and didcomm... if its dropped the URI becomes unresolvable, so that client would need to store that somewhere...The text was updated successfully, but these errors were encountered: