-
Notifications
You must be signed in to change notification settings - Fork 98
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Simplify abstract data model to be more concrete #855
Comments
chair hat off DID DHT does not use JSON-LD for extensibility for a few reasons:
I believe DID DHT could potentially be adjusted to add processing rules to transform the document to one with a context, and register LD term definitions alongside registered properties. That said, it would be a breaking change. I am curious how other DID Methods leverage the abstract data model, and it would be good to get a sense of the variety of implementations out there before seeing if it's feasible to define a concrete representation. Separately, I am not sure this type of change is permitted, as it might fall under the Class 4 definition:
Since I believe this could be considered a "new feature" by introducing new rules for representing DID Documents. |
I generally agree with the direction of simplifying the specification by removing the abstract data model and replacing it with a concrete one (which can then be converted to different representations like YaML, CBOR, etc.) |
I also agree that it is possible to remove the abstract data model in a way that does not affect existing implementation conformance and that we should make an attempt at doing this. To provide a concrete proposal, this would entail:
To be clear, if any of the steps above would result in a conforming DID Method becoming non-conformant, we'd clearly have to figure out how to fix the spec text so that doesn't happen. The goal here is to simplify the specification while not invalidating any currently conforming DID Methods. |
@decentralgabe wrote:
Hmm, the DID Core URL is 28 characters, a did:dht one would be maybe twice to three times that? Trading 75 characters for no deterministic way to do extensibility doesn't seem like a good trade off to me.
I don't understand these statements? IOW, the approach ensures that NO terms are defined (except for maybe in did:dht, and who knows if those definitions are going to conflict with definitions in other DID Methods). It feels like a recipe for guaranteed term conflicts in the future. I also don't understand "all terms have DNS-record mappings ahead of time" -- what does that mean? To be clear, I think |
I fully support a normative requirement on JSON-LD only core data model, and to eliminate the JSON and abstract data models from the next version of the technical recommendation. We've seen substantial confusion caused by this, and there is needless complexity and interoperability problems created by having an abstract data model, that is for the most part, just RDF... sometimes broken RDF. I think the W3C VCWG did the right thing, by clarifying that W3C VCs are always JSON-LD, and allowing alternative serialization of digital credentials such as ISO mDoc, OAUTH SD-JWTs, attribute certs and other formats to be developed elsewhere. I would recommend that the DID WG take a similar approach. Do JSON-LD based DIDS as well as they can be done at W3C. Do not attempt to define multiple serializations of the data model. Provide concrete resolution guidance based on the JSON-LD ecosystem, such as document loaders, which can handle either URNs or URLs, and which are already supported well in JSON-LD tooling. Address the If people want to do "did like things" in CBOR or YAML, let them do that... but make it clear that DIDs are JSON-LD, just like its now clear that W3C VCs are JSON-LD. |
@msporny it gets into the specifics of how did:dht works and there is more detail here but the short version is as a size saving mechanism the spec leverages a DID Document -> DNS Packet mapping, and then using DNS packet compression the result is saved on the DHT. We did an analysis of a number of compression formats (plain bytes, json, cbor, a custom binary serialization, and DNS) and found that DNS balanced an efficiency/already existing software tradeoff. Without a known mapping (or reverse mapping) between a property in the DID Doc and packet representation we cannot effectively store the record on the DHT, so these must be registered in the spec or a well known registry to reduce inconsistencies across implementations. The spec itself has a registry for this purpose. Leveraging the existing DID registry is likely the best process--noting properties supported by did:dht linked to their DID registry reference. This is the approach we've taken so far, but are open to other alternatives while maintaining the goal of saving as many bytes as possible. |
Ah, I see. I skimmed those sections and haven't tried to put the whole problem in my head to think about it more deeply. My gut reaction is that the "custom Domain-Specific Language for DNS encoding of DID Documents" thing feels a bit fraught, but that's a completely orthogonal issue. Based on what I saw in the spec, however, it feels like it would be fairly trivial for the DID Resolution process for
I would imagine that CBOR-LD applied to a In any case, with respect to changes to the abstract data model, I would expect that there wouldn't be an issue for |
Just to make it clear: this comment is with my W3C staff member's hat put down. TL;DR: my preference is to keep the abstract data model (ADM) as is. I have several reasons:
|
@wip-abramson Asked me to clarify my take on the abstract data model problem. He had suggested a different issue, but after re-reading this one, I think this is the better place to continue the thread. I have two main points. First, the ADM has proven to be a nightmare for interoperability. Instead of a concrete over-the-wire standard, we have a vague, confusing standard that requires dangerous warning signs like "Any interpretations of the specification (or its profiling specifications) that lead to different interpretations of JSON and JSON-LD, is either an implementation bug or a specification bug." The only reason we need this kind of guidance is because we failed to standardize a single over-the-wire representation for DID Documents. Second, the ADM should never have been in the last specification--and shouldn't be in this one--because there were not multiple implementations that tested it. As an untested feature, it should not have been allowed through to Recommendation. Rather than upset the apple cart at the time, I held my tongue to await the inevitable, expected objections from those < 1% of the organization that opposes our work. The fact is, an abstract data model is not testable. It must be made concrete to test it. There is some legitimate question about whether or not this is allowed under our current charter. Making the ADM testable by requiring a normative serialization would clarify a point of ambiguity in the existing standard, making this a legitimate Class 3 change. It would also let us stop calling it an Abstract Data Model and just call it a data model, with a normative serialization that we can actually test. It may be worth noting that during the re-chartering process, and in particular, in the resolution of the council in part due to my formal objection, I was assured that, if the WG wanted to clarify the ambiguity of the abstract data model, it falls within our scope. |
@jandrieu thanks for your comments.
I am curious how is this substantiated? I have generally seen no problems with interop and have myself been a part of a few different efforts across companies using various DID methods without issue. That's not to say there aren't issues, I just want to be clear about what they are.
It is your opinion that this is a failure. I have heard from a number of those involved in the ADM effort that it was a hard-fought compromise—and a success. Many of those involved have since left the group because they feel their work was done. Revising it without their presence is a dangerous practice.
The existing specification, as a published Recommendation with a test suite, has demonstrated sufficient testability to meet W3C requirements.
I would appreciate references here as I was unaware of this assurance. Regardless, I am less concerned with debating whether this is a Class 3 or Class 4 change, and more concerned with understanding whether this is a worthwhile use of the group's time, effort, especially given the substantial and inevitable knock-on effects. Unequivocally: I am strongly against this can of worms. Our chartered work on DID Resolution provides a more appropriate venue for addressing testability and interoperability concerns. I will heavily +1 @iherman's comment
I suggest we focus our group's limited resources on our chartered deliverables. We can explore ways to improve interoperability through DID Resolution specification work and some implementation guidance such as the context injection approach @msporny mentioned. Many have left the W3C work around VCs and DIDs because of our constant infighting. Spending substantial amounts of time and effort arguing about data formats is not fun, nor what standards work should be. The effect of this infighting has been pushing others out to different standards bodies. This isn't something that should be celebrated; it fractures our industry and gives our work, and way of working, an incredibly negative reputation. We have an opportunity to overcome that reputation, but that requires closing this issue, and moving forward. |
This was discussed during the #did meeting on 20 November 2024. View the transcriptw3c/did-core#855decentralgabe: The topic of today is issue 855, simplify abstract data model to be more concrete. decentralgabe: Manu opened the issue, Joe has made some comments, invite either one of you to queue to speak to the issue and what you'd like to see happen in the group. JoeAndrieu: I made my case in the Github issue, the abstract data model is a bad idea (my opinion), some think it's a good idea. My biggest problem is that its untestable and was not tested in last iteration, avoided raising that last time, political difficulties was why we avoided the discussion. We have spec features that are not implemented, we need to fix that. manu: I agree that the abstract data model doesnt buy us much in this spec decentralgabe: I want to highlight point about practical DID adoption - focus groups time on impactful time on objectives. markus_sabadello: I started with reason for abstract data model -- some wanted DID Documents in JSON-LD and plain JSON and that's obsolete already if we adopt controller document and I think we already had a resolution on one media type -- application/did -- both JSON and JSON-LD, interpretation would be the same whether there is a context, whether or not you use JSON-LD processor, interpretation is the same... with that, I'm thinking, why do we need an ivan: I was one arguing for abstract data model, still believe in it, that being said, I think most important point at this time is the controller document work -- my understanding, the DID Document and data model is a controller document with some minor features. Probably it's a bit extreme, but most of DID Core spec related to DID Document would be removed form standard anyway, in a sense, controller document might have made this entire discussion moot by now. decentralgabe: We might need more analysis on controller document impact? Is that a good next step? ChristopherA: I've been in favor of abstract model, in reality, it has now become a signal rather than a technical reality... there are a number of parties that will take removing it as a signal of 100% in on JSON-LD and that there are people will object to that, whether that's relevant or they're implementing DIDs, I don't know. Might cause us headaches. bigbluehat: Verifiable Credentials does JSON and JSON-LD without an abstract data model. manu: responding to ChristopherA, the text in current controller doc is not JSONLD maximalist. It says we would like you to include a context, but does not require it decentralgabe: A proposal to align DID Core with Controller Document to see what normative changes it might make and gauge reaction from community. markus_sabadello: I'm not so worried about political reactions, <Zakim> JoeAndrieu, you wanted to mention political pushback was based on a misunderstanding that JSON-LD is not JSON ivan: Yeah, I think I'll proposal something along the line of what Manu proposed: In view of controller document, and we decided to align, we can essentially declare the issue about abstract data model as moot -- no need to keep discussing, we will rely on controller document most of the time, whole thing will disappear without referring to it. It will just go away, automatically, we don't have to discuss it anymore, which creates energy loss. JoeAndrieu: That's interesting Ivan, I like that thinking, I do think as we shift to controller document, it has a concrete representation, we have to extract between two... won't make sense to make controller document abstract. JoeAndrieu: One of the reasons I'm not too concerned w/ political fallout, based on misunderstanding on JSON-LD vs. JSON -- people thought they were two different things, we can have common representation that both world views can accept current state. ChristopherA: If we had a non-CBOR-LD version of controller documents, boutique one, that we might show you can have other formats that we support. Don't see anyone on this call that is comitted to boutique CBOR representation, to demonstrate we're abstract instead of putting in the spec. Wip: The controller document argument is compelling, but this group needs to pass a resolution to adopt it, I think we were waiting for it to go into next stage of W3C Process. Wip: What is our story around allowing alternate formats? CBOR, YAML, etc? I think thing there is same as VCDM, people have to define bidirectional map from their representation into JSON. There should be a pathway for people to implement DID Documents in whatever way they want. decentralgabe: Chair hat off: Other representations, does it close the door on non-JSON representations, solved elsewhere? <Zakim> JoeAndrieu, you wanted to mention alternate formats <ivan> Issue on the reference to the Controller Document JoeAndrieu: I think it makes it easier, in current pattern, what we have is other representations -- go back to abstract data model, you have to go to abstract data model, which they didn't do in the first place. What we want is to go between serializations, anything is valid as long as you go from X to DID Document, then people can compress it ... it's just not a DID Document (yet), as long as you have a transformation that can get you back to the JoeAndrieu: If we start w/ abstract data model, it's more difficult, if we don't, it's easier to deal w/ alternate representations. <Zakim> manu, you wanted to comment on demonstrating CBOR manu: +1 to what JoeAndrieu & Wip said. We have language around being conformant with the DID document ecosystem. With bidirectional transformation rules. I agree with JoeAndrieu's point that without the ADM this is easier. As long as people can convert their representation into the DID doc representation you are golden. <ivan> Latest comment on the relationship to the CD manu: It is stable enough that the editors could make an attempt to alight the DID core spec with the controller document and remove the ADM. Seeing if we run into any normative changes that are required decentralgabe: We did pass resolution to align w/ controller document when we're ready. I'm good to not pass another one, just have Editor's do the work. If people feel that's not strong enough, we need a resolution, happy to run one if someone wants to draft the language. markus_sabadello: I agree with Manu and Joe, one thing I was wondering, does it affect extension properties in any way? it's a good thing to do that, how would that change if we don't have abstract data model? Would we require people to register their extensions in our registry? Still optional? FYI I do not see a resolution about adopting the controller document here - https://www.w3.org/2024/list-resolutions/?g=did burn: I added myself to queue before Manu -- this is basically an extended +1, about the direction we're talking about moving -- you had questioned this Markus, when I put first draft of spec into ReSpec, talked w/ Manu about this, I abstracted out from the specific syntax and try to create generic model and concrete realizations. burn: That was for two reasons -- it was very clear there was going to be a battle around JSON vs. JSON-LD -- this is the Verifiable Credentials spec -- other reason, we did want to make it clear to people was say that what we were trying to define wasn't limited to specific syntax, used XML as an example -- we've gradually removed XML and other things. This conversation has convinced me, as Manu said, there is another way to address challenges for original burn: As a proponent for the abstract data model (historically), I think arguments have been addressed correctly, we did same thing w/ DID spec because of what happened w/ VC spec, it's time to move on, thanks for all the great work! <ivan> Latest discussion in the group on the controller issue ChristopherA: General statements are: we can remove abstract syntax model if we do X, but then when I hear the list it includes things that are "oh, well it needs to be two way -- what does that mean?" Gordian can do elision and it will look just like a DID Document, conformant there, but Gordian-specific one that understands elision is a superset of capabilities and it isn't two way. One interpretation of two-way, along with this, we don't have to use ivan: I did some digging, I put pointers to minutes, last time we discussed this in WG, they all go in same direction - we didn't have a formal resolution, good to have it, agreement of WG to wait until CR. <Zakim> manu, you wanted to talk to registration of extensions manu: ChristopherA asked a good question. Wondering if we can defer answering what we mean by two way map until we are able to make a editorial pass. decentralgabe: Ok, I've updated the proposal, we need to figure out two-way transformations, other representations, and how extensions are managed. I would like to see the group agree to try to address these after, open to hear what everyone thinks. JoeAndrieu: I had proposed something incomplete, we don't control the controller document, we can't remove abstract data model from controller document -- a wrinkle -- proposal from Gabe is good, didn't have it depend on CR. I would also like to get into resolution on concrete representation for DID Document. decentralgabe: Maybe we can split into two resolutions? JoeAndrieu: I'm ok with two resolutions. decentralgabe: will run proposal after Markus. markus_sabadello: I agree with dealing w/ technical details after running resolution. <decentralgabe> PROPOSAL: Align DID Core with the Controller Document specification (https://www.w3.org/TR/controller-document/), and attempt to remove language around the Abstract Data Model. manu: +1 <burn> +1 <ivan> +1 <JoeAndrieu> +1 <markus_sabadello> +1 <Wip> +1 <TallTed> +1 <dmitriz> +1 <bigbluehat> +1 <danpape> +1 <JennieM> +1 <ChristopherA> +0 <ChristopherA> (pending other proposal) RESOLUTION: Align DID Core with the Controller Document specification (https://www.w3.org/TR/controller-document/), and attempt to remove language around the Abstract Data Model. <ChristopherA> reshare draft? decentralgabe: ok, on to the second proposal from Joe. ChristopherA: Is this the one where we can put in bidirectional transformation and interpretation of concrete representation, maybe? Extension and intent to support other formats. That's not in this proposal. JoeAndrieu: I'll try to wordsmith the proposal. manu: I think we need to be more specific. The controller document is very clear about it being JSON. The DID document should be just as clear <ivan> +1 to manu. We have to be aligned with the CD <burn> I suggest avoiding the word "serialization" and instead use "representation" or "syntax" manu: THis doesnt mean that it is the only representation. It is totally okay to have a CBOR representation. There are multiple levels. One is there is a JSON representation, if it follows all the rules you are good to go. The other is if you can get whatever representation you want into the concrete JSON representation then you are good to go <JoeAndrieu> good for me decentralgabe: Let's say we allow alternative representations and come up with that language later. markus_sabadello: Probably missing something, why do we need to profile this, why is this not part of profile document that we reference? JoeAndrieu: We don't control the controller document, we don't know what it'll end up as. markus_sabadello: We reference it, we will use it, profile it in a way -- why would profile have anything w/ representation -- JSON vs. JSON-LD? Controller document, how to use context, etc. JoeAndrieu: language i"m trying to get in -- expect other specs to profile controller document, serialization might be something we don't like. I don't think it'll go in that direction, but if we're dependent on that gorup, we have to profile it anyway, we have things in DID Core that have to do w/ that document. JoeAndrieu: If controller document changes, we would have to undo that in our profiling. <Zakim> manu, you wanted to note it has to do w/ media type. manu: I agree with JoeAndrieu. We dont currently specify a media type in the controller document. This has to do with media types. The DID spec will have a different media type to the controller document decentralgabe: Does that clarify, Markus? Ok with language? markus_sabadello: Sure, don't fully understand but not against, please run the proposal. <decentralgabe> PROPOSAL: When profiling the Controller Document for the DID Document specification we will use JSON as a concrete representation, with language describing options for alternative representations. <manu> +1 <JoeAndrieu> +1 <ivan> +1 <Wip> +1 <markus_sabadello> +0.5 <ChristopherA> +1 (and revises my previous +0 to +1) <bigbluehat> +1 <JennieM> +1 <danpape> +1 <decentralgabe> +1 <dmitriz> +1 RESOLUTION: When profiling the Controller Document for the DID Document specification we will use JSON as a concrete representation, with language describing options for alternative representations. decentralgabe: We need to ensure controller document has media type, we'll have to raise that issue over there. <ChristopherA> Ciao! |
It has been suggested that the abstract data model in DID Core creates unnecessary complexity and that a more concrete data model should be selected, based on implementation experience over the past two years. This issue is to track the discussion of how that simplification might occur.
The text was updated successfully, but these errors were encountered: