From 9632e3b5014d1324008429c3698c04d8125a814f Mon Sep 17 00:00:00 2001 From: Prad Nukala Date: Fri, 6 Dec 2024 13:01:48 -0500 Subject: [PATCH] docs(guide): add cosmos proto guide --- .github/aider/guides/cosmos-proto.md | 569 ++++++++++++ .github/aider/guides/cosmos-rfc.md | 627 +++++++++++++ .github/aider/guides/cosmos-sdk.md | 685 +++++++++++++++ .github/aider/guides/sonr-did.md | 141 +++ .github/aider/guides/sonr-dwn.md | 145 +++ .github/aider/guides/sonr-service.md | 91 ++ .github/aider/guides/sonr-token.md | 0 .github/aider/guides/ucan-spec.md | 873 +++++++++++++++++++ .github/aider/prompts/data-modeler-cosmos.md | 105 +++ .github/aider/prompts/data-modeler.md | 88 ++ .github/aider/prompts/sonr-tech-lead.md | 132 +++ README.md | 15 +- package.json | 3 +- 13 files changed, 3466 insertions(+), 8 deletions(-) create mode 100644 .github/aider/guides/cosmos-proto.md create mode 100644 .github/aider/guides/cosmos-rfc.md create mode 100644 .github/aider/guides/cosmos-sdk.md create mode 100644 .github/aider/guides/sonr-did.md create mode 100644 .github/aider/guides/sonr-dwn.md create mode 100644 .github/aider/guides/sonr-service.md create mode 100644 .github/aider/guides/sonr-token.md create mode 100644 .github/aider/guides/ucan-spec.md create mode 100644 .github/aider/prompts/data-modeler-cosmos.md create mode 100644 .github/aider/prompts/data-modeler.md create mode 100644 .github/aider/prompts/sonr-tech-lead.md diff --git a/.github/aider/guides/cosmos-proto.md b/.github/aider/guides/cosmos-proto.md new file mode 100644 index 0000000..9f44a52 --- /dev/null +++ b/.github/aider/guides/cosmos-proto.md @@ -0,0 +1,569 @@ +# Protocol Buffers in Cosmos SDK + +## Overview + +The Cosmos SDK uses Protocol Buffers for serialization and API definitions. Generation is handled via a Docker image: `ghcr.io/cosmos/proto-builder:0.15.x`. + +## Generation Tools + +- **Buf**: Primary tool for protobuf management +- **protocgen.sh**: Core generation script in `scripts/` +- **Makefile Commands**: Standard commands for generate, lint, format + +## Key Components + +### Buf Configuration + +1. **Workspace Setup** + - Root level buf workspace configuration + - Manages multiple protobuf directories + +2. **Directory Structure** + ``` + proto/ + ├── buf.gen.gogo.yaml # GoGo Protobuf generation + ├── buf.gen.pulsar.yaml # Pulsar API generation + ├── buf.gen.swagger.yaml # OpenAPI/Swagger docs + ├── buf.lock # Dependencies + ├── buf.yaml # Core configuration + ├── cosmos/ # Core protos + └── tendermint/ # Consensus protos + ``` + +3. **Module Protos** + - Located in `x/{moduleName}/proto` + - Module-specific message definitions + +#### `buf.gen.gogo.yaml` + +`buf.gen.gogo.yaml` defines how the protobuf files should be generated for use with in the module. This file uses [gogoproto](https://github.com/gogo/protobuf), a separate generator from the google go-proto generator that makes working with various objects more ergonomic, and it has more performant encode and decode steps + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/main/proto/buf.gen.gogo.yaml#L1-L9 +``` + +#### `buf.gen.pulsar.yaml` + +`buf.gen.pulsar.yaml` defines how protobuf files should be generated using the [new golang apiv2 of protobuf](https://go.dev/blog/protobuf-apiv2). This generator is used instead of the google go-proto generator because it has some extra helpers for Cosmos SDK applications and will have more performant encode and decode than the google go-proto generator. You can follow the development of this generator [here](https://github.com/cosmos/cosmos-proto). + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/main/proto/buf.gen.pulsar.yaml#L1-L18 +``` + +#### `buf.gen.swagger.yaml` + +`buf.gen.swagger.yaml` generates the swagger documentation for the query and messages of the chain. This will only define the REST API end points that were defined in the query and msg servers. You can find examples of this [here](https://github.com/cosmos/cosmos-sdk/blob/main/x/bank/proto/cosmos/bank/v1beta1/query.proto) + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/main/proto/buf.gen.swagger.yaml#L1-L6 +``` + +#### `buf.lock` + +This is an autogenerated file based off the dependencies required by the `.gen` files. There is no need to copy the current one. If you depend on cosmos-sdk proto definitions a new entry for the Cosmos SDK will need to be provided. The dependency you will need to use is `buf.build/cosmos/cosmos-sdk`. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/main/proto/buf.lock#L1-L16 +``` + +#### `buf.yaml` + +`buf.yaml` defines the [name of your package](https://github.com/cosmos/cosmos-sdk/blob/main/proto/buf.yaml#L3), which [breakage checker](https://buf.build/docs/tutorials/getting-started-with-buf-cli#detect-breaking-changes) to use and how to [lint your protobuf files](https://buf.build/docs/tutorials/getting-started-with-buf-cli#lint-your-api). + +It is advised to use a tagged version of the buf modules corresponding to the version of the Cosmos SDK being are used. + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/main/proto/buf.yaml#L1-L24 +``` + +We use a variety of linters for the Cosmos SDK protobuf files. The repo also checks this in ci. +A reference to the github actions can be found [here](https://github.com/cosmos/cosmos-sdk/blob/main/.github/workflows/proto.yml#L1-L32) + +# ORM + +The Cosmos SDK ORM is a state management library that provides a rich, but opinionated set of tools for managing a +module's state. It provides support for: + +- type safe management of state +- multipart keys +- secondary indexes +- unique indexes +- easy prefix and range queries +- automatic genesis import/export +- automatic query services for clients, including support for light client proofs (still in development) +- indexing state data in external databases (still in development) + +## Design and Philosophy + +The ORM's data model is inspired by the relational data model found in SQL databases. The core abstraction is a table +with a primary key and optional secondary indexes. + +Because the Cosmos SDK uses protobuf as its encoding layer, ORM tables are defined directly in .proto files using +protobuf options. Each table is defined by a single protobuf `message` type and a schema of multiple tables is +represented by a single .proto file. + +Table structure is specified in the same file where messages are defined in order to make it easy to focus on better +design of the state layer. Because blockchain state layout is part of the public API for clients (TODO: link to docs on +light client proofs), it is important to think about the state layout as being part of the public API of a module. +Changing the state layout actually breaks clients, so it is ideal to think through it carefully up front and to aim for +a design that will eliminate or minimize breaking changes down the road. Also, good design of state enables building +more performant and sophisticated applications. Providing users with a set of tools inspired by relational databases +which have a long history of database design best practices and allowing schema to be specified declaratively in a +single place are design choices the ORM makes to enable better design and more durable APIs. + +Also, by only supporting the table abstraction as opposed to key-value pair maps, it is easy to add to new +columns/fields to any data structure without causing a breaking change and the data structures can easily be indexed in +any off-the-shelf SQL database for more sophisticated queries. + +The encoding of fields in keys is designed to support ordered iteration for all protobuf primitive field types +except for `bytes` as well as the well-known types `google.protobuf.Timestamp` and `google.protobuf.Duration`. Encodings +are optimized for storage space when it makes sense (see the documentation in `cosmos/orm/v1/orm.proto` for more details) +and table rows do not use extra storage space to store key fields in the value. + +We recommend that users of the ORM attempt to follow database design best practices such as +[normalization](https://en.wikipedia.org/wiki/Database_normalization) (at least 1NF). +For instance, defining `repeated` fields in a table is considered an anti-pattern because breaks first normal form (1NF). +Although we support `repeated` fields in tables, they cannot be used as key fields for this reason. This may seem +restrictive but years of best practice (and also experience in the SDK) have shown that following this pattern +leads to easier to maintain schemas. + +To illustrate the motivation for these principles with an example from the SDK, historically balances were stored +as a mapping from account -> map of denom to amount. This did not scale well because an account with 100 token balances +needed to be encoded/decoded every time a single coin balance changed. Now balances are stored as account,denom -> amount +as in the example above. With the ORM's data model, if we wanted to add a new field to `Balance` such as +`unlocked_balance` (if vesting accounts were redesigned in this way), it would be easy to add it to this table without +requiring a data migration. Because of the ORM's optimizations, the account and denom are only stored in the key part +of storage and not in the value leading to both a flexible data model and efficient usage of storage. + +## Defining Tables + +To define a table: + +1. create a .proto file to describe the module's state (naming it `state.proto` is recommended for consistency), + and import "cosmos/orm/v1/orm.proto", ex: + +```protobuf +syntax = "proto3"; +package bank_example; + +import "cosmos/orm/v1/orm.proto"; +``` + +2. define a `message` for the table, ex: + +```protobuf +message Balance { + bytes account = 1; + string denom = 2; + uint64 balance = 3; +} +``` + +3. add the `cosmos.orm.v1.table` option to the table and give the table an `id` unique within this .proto file: + +```protobuf +message Balance { + option (cosmos.orm.v1.table) = { + id: 1 + }; + + bytes account = 1; + string denom = 2; + uint64 balance = 3; +} +``` + +4. define the primary key field or fields, as a comma-separated list of the fields from the message which should make + up the primary key: + +```protobuf +message Balance { + option (cosmos.orm.v1.table) = { + id: 1 + primary_key: { fields: "account,denom" } + }; + + bytes account = 1; + string denom = 2; + uint64 balance = 3; +} +``` + +5. add any desired secondary indexes by specifying an `id` unique within the table and a comma-separate list of the + index fields: + +```protobuf +message Balance { + option (cosmos.orm.v1.table) = { + id: 1; + primary_key: { fields: "account,denom" } + index: { id: 1 fields: "denom" } // this allows querying for the accounts which own a denom + }; + + bytes account = 1; + string denom = 2; + uint64 amount = 3; +} +``` + +### Auto-incrementing Primary Keys + +A common pattern in SDK modules and in database design is to define tables with a single integer `id` field with an +automatically generated primary key. In the ORM we can do this by setting the `auto_increment` option to `true` on the +primary key, ex: + +```protobuf +message Account { + option (cosmos.orm.v1.table) = { + id: 2; + primary_key: { fields: "id", auto_increment: true } + }; + + uint64 id = 1; + bytes address = 2; +} +``` + +### Unique Indexes + +A unique index can be added by setting the `unique` option to `true` on an index, ex: + +```protobuf +message Account { + option (cosmos.orm.v1.table) = { + id: 2; + primary_key: { fields: "id", auto_increment: true } + index: {id: 1, fields: "address", unique: true} + }; + + uint64 id = 1; + bytes address = 2; +} +``` + +### Singletons + +The ORM also supports a special type of table with only one row called a `singleton`. This can be used for storing +module parameters. Singletons only need to define a unique `id` and that cannot conflict with the id of other +tables or singletons in the same .proto file. Ex: + +```protobuf +message Params { + option (cosmos.orm.v1.singleton) = { + id: 3; + }; + + google.protobuf.Duration voting_period = 1; + uint64 min_threshold = 2; +} +``` + +## Running Codegen + +NOTE: the ORM will only work with protobuf code that implements the [google.golang.org/protobuf](https://pkg.go.dev/google.golang.org/protobuf) +API. That means it will not work with code generated using gogo-proto. + +To install the ORM's code generator, run: + +```shell +go install cosmossdk.io/orm/cmd/protoc-gen-go-cosmos-orm@latest +``` + +The recommended way to run the code generator is to use [buf build](https://docs.buf.build/build/usage). +This is an example `buf.gen.yaml` that runs `protoc-gen-go`, `protoc-gen-go-grpc` and `protoc-gen-go-cosmos-orm` +using buf managed mode: + +```yaml +version: v1 +managed: + enabled: true + go_package_prefix: + default: foo.bar/api # the go package prefix of your package + override: + buf.build/cosmos/cosmos-sdk: cosmossdk.io/api # required to import the Cosmos SDK api module +plugins: + - name: go + out: . + opt: paths=source_relative + - name: go-grpc + out: . + opt: paths=source_relative + - name: go-cosmos-orm + out: . + opt: paths=source_relative +``` + +## Using the ORM in a module + +### Initialization + +To use the ORM in a module, first create a `ModuleSchemaDescriptor`. This tells the ORM which .proto files have defined +an ORM schema and assigns them all a unique non-zero id. Ex: + +```go +var MyModuleSchema = &ormv1alpha1.ModuleSchemaDescriptor{ + SchemaFile: []*ormv1alpha1.ModuleSchemaDescriptor_FileEntry{ + { + Id: 1, + ProtoFileName: mymodule.File_my_module_state_proto.Path(), + }, + }, +} +``` + +In the ORM generated code for a file named `state.proto`, there should be an interface `StateStore` that got generated +with a constructor `NewStateStore` that takes a parameter of type `ormdb.ModuleDB`. Add a reference to `StateStore` +to your module's keeper struct. Ex: + +```go +type Keeper struct { + db StateStore +} +``` + +Then instantiate the `StateStore` instance via an `ormdb.ModuleDB` that is instantiated from the `SchemaDescriptor` +above and one or more store services from `cosmossdk.io/core/store`. Ex: + +```go +func NewKeeper(storeService store.KVStoreService) (*Keeper, error) { + modDb, err := ormdb.NewModuleDB(MyModuleSchema, ormdb.ModuleDBOptions{KVStoreService: storeService}) + if err != nil { + return nil, err + } + db, err := NewStateStore(modDb) + if err != nil { + return nil, err + } + return Keeper{db: db}, nil +} +``` + +### Using the generated code + +The generated code for the ORM contains methods for inserting, updating, deleting and querying table entries. +For each table in a .proto file, there is a type-safe table interface implemented in generated code. For instance, +for a table named `Balance` there should be a `BalanceTable` interface that looks like this: + +```go +type BalanceTable interface { + Insert(ctx context.Context, balance *Balance) error + Update(ctx context.Context, balance *Balance) error + Save(ctx context.Context, balance *Balance) error + Delete(ctx context.Context, balance *Balance) error + Has(ctx context.Context, account []byte, denom string) (found bool, err error) + // Get returns nil and an error which responds true to ormerrors.IsNotFound() if the record was not found. + Get(ctx context.Context, account []byte, denom string) (*Balance, error) + List(ctx context.Context, prefixKey BalanceIndexKey, opts ...ormlist.Option) (BalanceIterator, error) + ListRange(ctx context.Context, from, to BalanceIndexKey, opts ...ormlist.Option) (BalanceIterator, error) + DeleteBy(ctx context.Context, prefixKey BalanceIndexKey) error + DeleteRange(ctx context.Context, from, to BalanceIndexKey) error + + doNotImplement() +} +``` + +This `BalanceTable` should be accessible from the `StateStore` interface (assuming our file is named `state.proto`) +via a `BalanceTable()` accessor method. If all the above example tables/singletons were in the same `state.proto`, +then `StateStore` would get generated like this: + +```go +type BankStore interface { + BalanceTable() BalanceTable + AccountTable() AccountTable + ParamsTable() ParamsTable + + doNotImplement() +} +``` + +So to work with the `BalanceTable` in a keeper method we could use code like this: + +```go +func (k keeper) AddBalance(ctx context.Context, acct []byte, denom string, amount uint64) error { + balance, err := k.db.BalanceTable().Get(ctx, acct, denom) + if err != nil && !ormerrors.IsNotFound(err) { + return err + } + + if balance == nil { + balance = &Balance{ + Account: acct, + Denom: denom, + Amount: amount, + } + } else { + balance.Amount = balance.Amount + amount + } + + return k.db.BalanceTable().Save(ctx, balance) +} +``` + +`List` methods take `IndexKey` parameters. For instance, `BalanceTable.List` takes `BalanceIndexKey`. `BalanceIndexKey` +let's represent index keys for the different indexes (primary and secondary) on the `Balance` table. The primary key +in the `Balance` table gets a struct `BalanceAccountDenomIndexKey` and the first index gets an index key `BalanceDenomIndexKey`. +If we wanted to list all the denoms and amounts that an account holds, we would use `BalanceAccountDenomIndexKey` +with a `List` query just on the account prefix. Ex: + +```go +it, err := keeper.db.BalanceTable().List(ctx, BalanceAccountDenomIndexKey{}.WithAccount(acct)) +``` + +--- + +## sidebar_position: 1 + +# ProtocolBuffer Annotations + +This document explains the various protobuf scalars that have been added to make working with protobuf easier for Cosmos SDK application developers + +## Signer + +Signer specifies which field should be used to determine the signer of a message for the Cosmos SDK. This field can be used for clients as well to infer which field should be used to determine the signer of a message. + +Read more about the signer field [here](./02-messages-and-queries.md). + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/e6848d99b55a65d014375b295bdd7f9641aac95e/proto/cosmos/bank/v1beta1/tx.proto#L40 +``` + +```proto +option (cosmos.msg.v1.signer) = "from_address"; +``` + +## Scalar + +The scalar type defines a way for clients to understand how to construct protobuf messages according to what is expected by the module and sdk. + +```proto +(cosmos_proto.scalar) = "cosmos.AddressString" +``` + +Example of account address string scalar: + +```proto reference +https://github.com/cosmos/cosmos-sdk/blob/e6848d99b55a65d014375b295bdd7f9641aac95e/proto/cosmos/bank/v1beta1/tx.proto#L46 +``` + +Example of validator address string scalar: + +```proto reference +https://github.com/cosmos/cosmos-sdk/blob/e8f28bf5db18b8d6b7e0d94b542ce4cf48fed9d6/proto/cosmos/distribution/v1beta1/query.proto#L87 +``` + +Example of pubkey scalar: + +```proto reference +https://github.com/cosmos/cosmos-sdk/blob/11068bfbcd44a7db8af63b6a8aa079b1718f6040/proto/cosmos/staking/v1beta1/tx.proto#L94 +``` + +Example of Decimals scalar: + +```proto reference +https://github.com/cosmos/cosmos-sdk/blob/e8f28bf5db18b8d6b7e0d94b542ce4cf48fed9d6/proto/cosmos/distribution/v1beta1/distribution.proto#L26 +``` + +Example of Int scalar: + +```proto reference +https://github.com/cosmos/cosmos-sdk/blob/e8f28bf5db18b8d6b7e0d94b542ce4cf48fed9d6/proto/cosmos/gov/v1/gov.proto#L137 +``` + +There are a few options for what can be provided as a scalar: `cosmos.AddressString`, `cosmos.ValidatorAddressString`, `cosmos.ConsensusAddressString`, `cosmos.Int`, `cosmos.Dec`. + +## Implements_Interface + +Implement interface is used to provide information to client tooling like [telescope](https://github.com/cosmology-tech/telescope) on how to encode and decode protobuf messages. + +```proto +option (cosmos_proto.implements_interface) = "cosmos.auth.v1beta1.AccountI"; +``` + +## Method,Field,Message Added In + +`method_added_in`, `field_added_in` and `message_added_in` are annotations to denotate to clients that a field has been supported in a later version. This is useful when new methods or fields are added in later versions and that the client needs to be aware of what it can call. + +The annotation should be worded as follow: + +```proto +option (cosmos_proto.method_added_in) = "cosmos-sdk v0.50.1"; +option (cosmos_proto.method_added_in) = "x/epochs v1.0.0"; +option (cosmos_proto.method_added_in) = "simapp v24.0.0"; +``` + +## Amino + +The amino codec was removed in `v0.50+`, this means there is not a need register `legacyAminoCodec`. To replace the amino codec, Amino protobuf annotations are used to provide information to the amino codec on how to encode and decode protobuf messages. + +:::note +Amino annotations are only used for backwards compatibility with amino. New modules are not required use amino annotations. +::: + +The below annotations are used to provide information to the amino codec on how to encode and decode protobuf messages in a backwards compatible manner. + +### Name + +Name specifies the amino name that would show up for the user in order for them see which message they are signing. + +```proto +option (amino.name) = "cosmos-sdk/BaseAccount"; +``` + +```proto reference +https://github.com/cosmos/cosmos-sdk/blob/e8f28bf5db18b8d6b7e0d94b542ce4cf48fed9d6/proto/cosmos/bank/v1beta1/tx.proto#L41 +``` + +### Field_Name + +Field name specifies the amino name that would show up for the user in order for them see which field they are signing. + +```proto +uint64 height = 1 [(amino.field_name) = "public_key"]; +``` + +```proto reference +https://github.com/cosmos/cosmos-sdk/blob/e8f28bf5db18b8d6b7e0d94b542ce4cf48fed9d6/proto/cosmos/distribution/v1beta1/distribution.proto#L166 +``` + +### Dont_OmitEmpty + +Dont omitempty specifies that the field should not be omitted when encoding to amino. + +```proto +repeated cosmos.base.v1beta1.Coin amount = 3 [(amino.dont_omitempty) = true]; +``` + +```proto reference +https://github.com/cosmos/cosmos-sdk/blob/e8f28bf5db18b8d6b7e0d94b542ce4cf48fed9d6/proto/cosmos/bank/v1beta1/bank.proto#L56 +``` + +### Encoding + +Encoding instructs the amino json marshaler how to encode certain fields that may differ from the standard encoding behaviour. The most common example of this is how `repeated cosmos.base.v1beta1.Coin` is encoded when using the amino json encoding format. The `legacy_coins` option tells the json marshaler [how to encode a null slice](https://github.com/cosmos/cosmos-sdk/blob/e8f28bf5db18b8d6b7e0d94b542ce4cf48fed9d6/x/tx/signing/aminojson/json_marshal.go#L65) of `cosmos.base.v1beta1.Coin`. + +```proto +(amino.encoding) = "legacy_coins", +``` + +```proto reference +https://github.com/cosmos/cosmos-sdk/blob/e8f28bf5db18b8d6b7e0d94b542ce4cf48fed9d6/proto/cosmos/bank/v1beta1/genesis.proto#L23 +``` + +Another example is a protobuf `bytes` that contains a valid JSON document. +The `inline_json` option tells the json marshaler to embed the JSON bytes into the wrapping document without escaping. + +```proto +(amino.encoding) = "inline_json", +``` + +E.g. the bytes containing `{"foo":123}` in the `envelope` field would lead to the following JSON: + +```json +{ + "envelope": { + "foo": 123 + } +} +``` + +If the bytes are not valid JSON, this leads to JSON broken documents. Thus a JSON validity check needs to be in place at some point of the process. diff --git a/.github/aider/guides/cosmos-rfc.md b/.github/aider/guides/cosmos-rfc.md new file mode 100644 index 0000000..d1b9704 --- /dev/null +++ b/.github/aider/guides/cosmos-rfc.md @@ -0,0 +1,627 @@ +# RFC 004: Account System Refactor + +## Status +- Draft v2 (May 2023) + +## Current Limitations + +1. **Account Representation**: Limited by `google.Protobuf.Any` encapsulation and basic authentication methods +2. **Interface Constraints**: Lacks support for advanced functionalities like vesting and complex auth systems +3. **Implementation Rigidity**: Poor differentiation between account types (e.g., `ModuleAccount`) +4. **Authorization System**: Basic `x/auth` module with limited scope beyond `x/bank` functionality +5. **Dependency Issues**: Cyclic dependencies between modules (e.g., `x/auth` ↔ `x/bank` for vesting) + +## Proposal + +This proposal aims to transform the way accounts are managed within the Cosmos SDK by introducing significant changes to +their structure and functionality. + +### Rethinking Account Representation and Business Logic + +Instead of representing accounts as simple `google.Protobuf.Any` structures stored in state with no business logic +attached, this proposal suggests a more sophisticated account representation that is closer to module entities. +In fact, accounts should be able to receive messages and process them in the same way modules do, and be capable of storing +state in a isolated (prefixed) portion of state belonging only to them, in the same way as modules do. + +### Account Message Reception + +We propose that accounts should be able to receive messages in the same way modules can, allowing them to manage their +own state modifications without relying on other modules. This change would enable more advanced account functionality, such as the +`VestingAccount` example, where the x/bank module previously needed to change the vestingState by casting the abstracted +account to `VestingAccount` and triggering the `TrackDelegation` call. Accounts are already capable of sending messages when +a state transition, originating from a transaction, is executed. + +When accounts receive messages, they will be able to identify the sender of the message and decide how to process the +state transition, if at all. + +### Consequences + +These changes would have significant implications for the Cosmos SDK, resulting in a system of actors that are equal from +the runtime perspective. The runtime would only be responsible for propagating messages between actors and would not +manage the authorization system. Instead, actors would manage their own authorizations. For instance, there would be no +need for the `x/auth` module to manage minting or burning of coins permissions, as it would fall within the scope of the +`x/bank` module. + +The key difference between accounts and modules would lie in the origin of the message (state transition). Accounts +(ExternallyOwnedAccount), which have credentials (e.g., a public/private key pairing), originate state transitions from +transactions. In contrast, module state transitions do not have authentication credentials backing them and can be +caused by two factors: either as a consequence of a state transition coming from a transaction or triggered by a scheduler +(e.g., the runtime's Begin/EndBlock). + +By implementing these proposed changes, the Cosmos SDK will benefit from a more extensible, versatile, and efficient account +management system that is better suited to address the requirements of the Cosmos ecosystem. + +#### Standardization + +With `x/accounts` allowing a modular api there becomes a need for standardization of accounts or the interfaces wallets and other clients should expect to use. For this reason we will be using the [`CIP` repo](https://github.com/cosmos/cips) in order to standardize interfaces in order for wallets to know what to expect when interacting with accounts. + +## Implementation + +### Account Definition + +We define the new `Account` type, which is what an account needs to implement to be treated as such. +An `Account` type is defined at APP level, so it cannot be dynamically loaded as the chain is running without upgrading the +node code, unless we create something like a `CosmWasmAccount` which is an account backed by an `x/wasm` contract. + +```go +// Account is what the developer implements to define an account. +type Account[InitMsg proto.Message] interface { + // Init is the function that initialises an account instance of a given kind. + // InitMsg is used to initialise the initial state of an account. + Init(ctx *Context, msg InitMsg) error + // RegisterExecuteHandlers registers an account's execution messages. + RegisterExecuteHandlers(executeRouter *ExecuteRouter) + // RegisterQueryHandlers registers an account's query messages. + RegisterQueryHandlers(queryRouter *QueryRouter) + // RegisterMigrationHandlers registers an account's migration messages. + RegisterMigrationHandlers(migrationRouter *MigrationRouter) +} +``` + +### The InternalAccount definition + +The public `Account` interface implementation is then converted by the runtime into an `InternalAccount` implementation, +which contains all the information and business logic needed to operate the account. + +```go +type Schema struct { + state StateSchema // represents the state of an account + init InitSchema // represents the init msg schema + exec ExecSchema // represents the multiple execution msg schemas, containing also responses + query QuerySchema // represents the multiple query msg schemas, containing also responses + migrate *MigrateSchema // represents the multiple migrate msg schemas, containing also responses, it's optional +} + +type InternalAccount struct { + init func(ctx *Context, msg proto.Message) (*InitResponse, error) + execute func(ctx *Context, msg proto.Message) (*ExecuteResponse, error) + query func(ctx *Context, msg proto.Message) (proto.Message, error) + schema func() *Schema + migrate func(ctx *Context, msg proto.Message) (*MigrateResponse, error) +} +``` + +This is an internal view of the account as intended by the system. It is not meant to be what developers implement. An +example implementation of the `InternalAccount` type can be found in [this](https://github.com/testinginprod/accounts-poc/blob/main/examples/recover/recover.go) +example of account whose credentials can be recovered. In fact, even if the `Internal` implementation is untyped (with +respect to `proto.Message`), the concrete implementation is fully typed. + +During any of the execution methods of `InternalAccount`, `schema` excluded, the account is given a `Context` which provides: + +- A namespaced `KVStore` for the account, which isolates the account state from others (NOTE: no `store keys` needed, + the account address serves as `store key`). +- Information regarding itself (its address) +- Information regarding the sender. +- ... + +#### Init + +Init defines the entrypoint that allows for a new account instance of a given kind to be initialised. +The account is passed some opaque protobuf message which is then interpreted and contains the instructions that +constitute the initial state of an account once it is deployed. + +An `Account` code can be deployed multiple times through the `Init` function, similar to how a `CosmWasm` contract code +can be deployed (Instantiated) multiple times. + +#### Execute + +Execute defines the entrypoint that allows an `Account` to process a state transition, the account can decide then how to +process the state transition based on the message provided and the sender of the transition. + +#### Query + +Query defines a read-only entrypoint that provides a stable interface that links an account with its state. The reason for +which `Query` is still being preferred as an addition to raw state reflection is to: + +- Provide a stable interface for querying (state can be optimised and change more frequently than a query) +- Provide a way to define an account `Interface` with respect to its `Read/Write` paths. +- Provide a way to query information that cannot be processed from raw state reflection, ex: compute information from lazy + state that has not been yet concretely processed (eg: balances with respect to lazy inputs/outputs) + +#### Schema + +Schema provides the definition of an account from `API` perspective, and it's the only thing that should be taken into account +when interacting with an account from another account or module, for example: an account is an `authz-interface` account if +it has the following message in its execution messages `MsgProxyStateTransition{ state_transition: google.Protobuf.Any }`. + +### Migrate + +Migrate defines the entrypoint that allows an `Account` to migrate its state from a previous version to a new one. Migrations +can be initiated only by the account itself, concretely this means that the migrate action sender can only be the account address +itself, if the account wants to allow another address to migrate it on its behalf then it could create an execution message +that makes the account migrate itself. + +### x/accounts module + +In order to create accounts we define a new module `x/accounts`, note that `x/accounts` deploys account with no authentication +credentials attached to it which means no action of an account can be incepted from a TX, we will later explore how the +`x/authn` module uses `x/accounts` to deploy authenticated accounts. + +This also has another important implication for which account addresses are now fully decoupled from the authentication mechanism +which makes in turn off-chain operations a little more complex, as the chain becomes the real link between account identifier +and credentials. + +We could also introduce a way to deterministically compute the account address. + +Note, from the transaction point of view, the `init_message` and `execute_message` are opaque `google.Protobuf.Any`. + +The module protobuf definition for `x/accounts` are the following: + +```protobuf +// Msg defines the Msg service. +service Msg { + rpc Deploy(MsgDeploy) returns (MsgDeployResponse); + rpc Execute(MsgExecute) returns (MsgExecuteResponse); + rpc Migrate(MsgMigrate) returns (MsgMigrateResponse); +} + +message MsgDeploy { + string sender = 1; + string kind = 2; + google.Protobuf.Any init_message = 3; + repeated google.Protobuf.Any authorize_messages = 4 [(gogoproto.nullable) = false]; +} + +message MsgDeployResponse { + string address = 1; + uint64 id = 2; + google.Protobuf.Any data = 3; +} + +message MsgExecute { + string sender = 1; + string address = 2; + google.Protobuf.Any message = 3; + repeated google.Protobuf.Any authorize_messages = 4 [(gogoproto.nullable) = false]; +} + +message MsgExecuteResponse { + google.Protobuf.Any data = 1; +} + +message MsgMigrate { + string sender = 1; + string new_account_kind = 2; + google.Protobuf.Any migrate_message = 3; +} + +message MsgMigrateResponse { + google.Protobuf.Any data = 1; +} + +``` + +#### MsgDeploy + +Deploys a new instance of the given account `kind` with initial settings represented by the `init_message` which is a `google.Protobuf.Any`. +Of course the `init_message` can be empty. A response is returned containing the account ID and humanised address, alongside some response +that the account instantiation might produce. + +#### Address derivation + +In order to decouple public keys from account addresses, we introduce a new address derivation mechanism which is + +#### MsgExecute + +Sends a `StateTransition` execution request, where the state transition is represented by the `message` which is a `google.Protobuf.Any`. +The account can then decide if to process it or not based on the `sender`. + +### MsgMigrate + +Migrates an account to a new version of itself, the new version is represented by the `new_account_kind`. The state transition +can only be incepted by the account itself, which means that the `sender` must be the account address itself. During the migration +the account current state is given to the new version of the account, which then executes the migration logic using the `migrate_message`, +it might change state or not, it's up to the account to decide. The response contains possible data that the account might produce +after the migration. + +#### Authorize Messages + +The `Deploy` and `Execute` messages have a field in common called `authorize_messages`, these messages are messages that the account +can execute on behalf of the sender. For example, in case an account is expecting some funds to be sent from the sender, +the sender can attach a `MsgSend` that the account can execute on the sender's behalf. These authorizations are short-lived, +they live only for the duration of the `Deploy` or `Execute` message execution, or until they are consumed. + +An alternative would have been to add a `funds` field, like it happens in cosmwasm, which guarantees the called contract that +the funds are available and sent in the context of the message execution. This would have been a simpler approach, but it would +have been limited to the context of `MsgSend` only, where the asset is `sdk.Coins`. The proposed generic way, instead, allows +the account to execute any message on behalf of the sender, which is more flexible, it could include NFT send execution, or +more complex things like `MsgMultiSend` or `MsgDelegate`, etc. + +### Further discussion + +#### Sub-accounts + +We could provide a way to link accounts to other accounts. Maybe during deployment the sender could decide to link the +newly created to its own account, although there might be use-cases for which the deployer is different from the account +that needs to be linked, in this case a handshake protocol on linking would need to be defined. + +#### Predictable address creation + +We need to provide a way to create an account with a predictable address, this might serve a lot of purposes, like accounts +wanting to generate an address that: + +- nobody else can claim besides the account used to generate the new account +- is predictable + +For example: + +```protobuf + +message MsgDeployPredictable { + string sender = 1; + uint32 nonce = 2; + ... +} +``` + +And then the address becomes `bechify(concat(sender, nonce))` + +`x/accounts` would still use the monotonically increasing sequence as account number. + +#### Joining Multiple Accounts + +As developers are building new kinds of accounts, it becomes necessary to provide a default way to combine the +functionalities of different account types. This allows developers to avoid duplicating code and enables end-users to +create or migrate to accounts with multiple functionalities without requiring custom development. + +To address this need, we propose the inclusion of a default account type called "MultiAccount". The MultiAccount type is +designed to merge the functionalities of other accounts by combining their execution, query, and migration APIs. +The account joining process would only fail in the case of API (intended as non-state Schema APIs) conflicts, ensuring +compatibility and consistency. + +With the introduction of the MultiAccount type, users would have the option to either migrate their existing accounts to +a MultiAccount type or extend an existing MultiAccount with newer APIs. This flexibility empowers users to leverage +various account functionalities without compromising compatibility or resorting to manual code duplication. + +The MultiAccount type serves as a standardized solution for combining different account functionalities within the +cosmos-sdk ecosystem. By adopting this approach, developers can streamline the development process and users can benefit +from a modular and extensible account system. + +# ADR 071: Cryptography v2- Multi-curve support + +## Change log + +- May 7th 2024: Initial Draft (Zondax AG: @raynaudoe @juliantoledano @jleni @educlerici-zondax @lucaslopezf) +- June 13th 2024: Add CometBFT implementation proposal (Zondax AG: @raynaudoe @juliantoledano @jleni @educlerici-zondax @lucaslopezf) +- July 2nd 2024: Split ADR proposal, add link to ADR in cosmos/crypto (Zondax AG: @raynaudoe @juliantoledano @jleni @educlerici-zondax @lucaslopezf) + +## Status + +DRAFT + +## Abstract + +This ADR proposes the refactoring of the existing `Keyring` and `cosmos-sdk/crypto` code to implement [ADR-001-CryptoProviders](https://github.com/cosmos/crypto/blob/main/docs/architecture/adr-001-crypto-provider.md). + +For in-depth details of the `CryptoProviders` and their design please refer to ADR mentioned above. + +## Introduction + +The introduction of multi-curve support in the cosmos-sdk cryptographic package offers significant advantages. By not being restricted to a single cryptographic curve, developers can choose the most appropriate curve based on security, performance, and compatibility requirements. This flexibility enhances the application's ability to adapt to evolving security standards and optimizes performance for specific use cases, helping to future-proofing the sdk's cryptographic capabilities. + +The enhancements in this proposal not only render the ["Keyring ADR"](https://github.com/cosmos/cosmos-sdk/issues/14940) obsolete, but also encompass its key aspects, replacing it with a more flexible and comprehensive approach. Furthermore, the gRPC service proposed in the mentioned ADR can be easily implemented as a specialized `CryptoProvider`. + +### Glossary + +1. **Interface**: In the context of this document, "interface" refers to Go's interface. + +2. **Module**: In this document, "module" refers to a Go module. + +3. **Package**: In the context of Go, a "package" refers to a unit of code organization. + +## Context + +In order to fully understand the need for changes and the proposed improvements, it's crucial to consider the current state of affairs: + +- The Cosmos SDK currently lacks a comprehensive ADR for the cryptographic package. + +- If a blockchain project requires a cryptographic curve that is not supported by the current SDK, the most likely scenario is that they will need to fork the SDK repository and make modifications. These modifications could potentially make the fork incompatible with future updates from the upstream SDK, complicating maintenance and integration. + +- Type leakage of specific crypto data types expose backward compatibility and extensibility challenges. + +- The demand for a more flexible and extensible approach to cryptography and address management is high. + +- Architectural changes are necessary to resolve many of the currently open issues related to new curves support. + +- There is a current trend towards modularity in the Interchain stack (e.g., runtime modules). + +- Security implications are a critical consideration during the redesign work. + +## Objectives + +The key objectives for this proposal are: + +- Leverage `CryptoProviders`: Utilize them as APIs for cryptographic tools, ensuring modularity, flexibility, and ease of integration. + +Developer-Centric Approach + +- Prioritize clear, intuitive interfaces and best-practice design principles. + +Quality Assurance + +- Enhanced Test Coverage: Improve testing methodologies to ensure the robustness and reliability of the module. + +## Technical Goals + +New Keyring: + +- Design a new `Keyring` interface with modular backends injection system to support hardware devices and cloud-based HSMs. This feature is optional and tied to complexity; if it proves too complex, it will be deferred to a future release as an enhancement. + +## Proposed architecture + +### Components + +The main components to be used will be the same as those found in the [ADR-001](https://github.com/cosmos/crypto/blob/main/docs/architecture/adr-001-crypto-provider.md#components). + +#### Storage and persistence + +The storage and persistence layer is tasked with storing a `CryptoProvider`s. Specifically, this layer must: + +- Securely store the crypto provider's associated private key (only if stored locally, otherwise a reference to the private key will be stored instead). +- Store the [`ProviderMetadata`](https://github.com/cosmos/crypto/blob/main/docs/architecture/adr-001-crypto-provider.md#metadata) struct which contains the data that distinguishes that provider. + +The purpose of this layer is to ensure that upon retrieval of the persisted data, we can access the provider's type, version, and specific configuration (which varies based on the provider type). This information will subsequently be utilized to initialize the appropriate factory, as detailed in the following section on the factory pattern. + +The storage proposal involves using a modified version of the [Record](https://github.com/cosmos/cosmos-sdk/blob/main/proto/cosmos/crypto/keyring/v1/record.proto) struct, which is already defined in **Keyring/v1**. Additionally, we propose utilizing the existing keyring backends (keychain, filesystem, memory, etc.) to store these `Record`s in the same manner as the current **Keyring/v1**. + +_Note: This approach will facilitate a smoother migration path from the current Keyring/v1 to the proposed architecture._ + +Below is the proposed protobuf message to be included in the modified `Record.proto` file + +##### Protobuf message structure + +The [record.proto](https://github.com/cosmos/cosmos-sdk/blob/main/proto/cosmos/crypto/keyring/v1/record.proto) file will be modified to include the `CryptoProvider` message as an optional field as follows. + +```protobuf + +// record.proto + +message Record { + string name = 1; + google.protobuf.Any pub_key = 2; + + oneof item { + Local local = 3; + Ledger ledger = 4; + Multi multi = 5; + Offline offline = 6; + CryptoProvider crypto_provider = 7; // <- New + } + + message Local { + google.protobuf.Any priv_key = 1; + } + + message Ledger { + hd.v1.BIP44Params path = 1; + } + + message Multi {} + + message Offline {} +} +``` + +##### Creating and loading a `CryptoProvider` + +For creating providers, we propose a _factory pattern_ and a _registry_ for these builders. Examples of these +patterns can be found [here](https://github.com/cosmos/crypto/blob/main/docs/architecture/adr-001-crypto-provider.md#illustrative-code-snippets) + +##### Keyring + +The new `Keyring` interface will serve as a central hub for managing and fetching `CryptoProviders`. To ensure a smoother migration path, the new Keyring will be backward compatible with the previous version. Since this will be the main API from which applications will obtain their `CryptoProvider` instances, the proposal is to extend the Keyring interface to include the methods: + +```go +type KeyringV2 interface { + // methods from Keyring/v1 + + // ListCryptoProviders returns a list of all the stored CryptoProvider metadata. + ListCryptoProviders() ([]ProviderMetadata, error) + + // GetCryptoProvider retrieves a specific CryptoProvider by its id. + GetCryptoProvider(id string) (CryptoProvider, error) +} +``` + +_Note_: Methods to obtain a provider from a public key or other means that make it easier to load the desired provider can be added. + +##### Especial use case: remote signers + +It's important to note that the `CryptoProvider` interface is versatile enough to be implemented as a remote signer. This capability allows for the integration of remote cryptographic operations, which can be particularly useful in distributed or cloud-based environments where local cryptographic resources are limited or need to be managed centrally. + +## Alternatives + +It is important to note that all the code presented in this document is not in its final form and could be subject to changes at the time of implementation. The examples and implementations discussed should be interpreted as alternatives, providing a conceptual framework rather than definitive solutions. This flexibility allows for adjustments based on further insights, technical evaluations, or changing requirements as development progresses. + +## Decision + +We will: + +- Leverage crypto providers +- Refactor the module structure as described above. +- Define types and interfaces as the code attached. +- Refactor existing code into new structure and interfaces. +- Implement Unit Tests to ensure no backward compatibility issues. + +## Consequences + +### Impact on the SDK codebase + +We can divide the impact of this ADR into two main categories: state machine code and client related code. + +#### Client + +The major impact will be on the client side, where the current `Keyring` interface will be replaced by the new `KeyringV2` interface. At first, the impact will be low since `CryptoProvider` is an optional field in the `Record` message, so there's no mandatory requirement for migrating to this new concept right away. This allows a progressive transition where the risks of breaking changes or regressions are minimized. + +#### State Machine + +The impact on the state machine code will be minimal, the modules affected (at the time of writing this ADR) +are the `x/accounts` module, specifically the `Authenticate` function and the `x/auth/ante` module. This function will need to be adapted to use a `CryptoProvider` service to make use of the `Verifier` instance. + +Worth mentioning that there's also the alternative of using `Verifier` instances in a standalone fashion (see note below). + +The specific way to adapt these modules will be deeply analyzed and decided at implementation time of this ADR. + +_Note_: All cryptographic tools (hashers, verifiers, signers, etc.) will continue to be available as standalone packages that can be imported and utilized directly without the need for a `CryptoProvider` instance. However, the `CryptoProvider` is the recommended method for using these tools as it offers a more secure way to handle sensitive data, enhanced modularity, and the ability to store configurations and metadata within the `CryptoProvider` definition. + +### Backwards Compatibility + +The proposed migration path is similar to what the cosmos-sdk has done in the past. To ensure a smooth transition, the following steps will be taken: + +Once ADR-001 is implemented with a stable release: + +- Deprecate the old crypto package. The old crypto package will still be usable, but it will be marked as deprecated and users can opt to use the new package. +- Migrate the codebase to use the new cosmos/crypto package and remove the old crypto one. + +### Positive + +- Single place of truth +- Easier to use interfaces +- Easier to extend +- Unit test for each crypto package +- Greater maintainability +- Incentivize addition of implementations instead of forks +- Decoupling behavior from implementation +- Sanitization of code + +### Negative + +- It will involve an effort to adapt existing code. +- It will require attention to detail and audition. + +### Neutral + +- It will involve extensive testing. + +## Test Cases + +- The code will be unit tested to ensure a high code coverage +- There should be integration tests around Keyring and CryptoProviders. + +> While an ADR is in the DRAFT or PROPOSED stage, this section should contain a +> summary of issues to be solved in future iterations (usually referencing comments +> from a pull-request discussion). +> +> Later, this section can optionally list ideas or improvements the author or +> reviewers found during the analysis of this ADR. + +# ADR-71 Bank V2 + +## Status + +DRAFT + +## Changelog + +- 2024-05-08: Initial Draft (@samricotta, @julienrbrt) + +## Abstract + +The primary objective of refactoring the bank module is to simplify and enhance the functionality of the Cosmos SDK. Over time the bank module has been burdened with numerous responsibilities including transaction handling, account restrictions, delegation counting, and the minting and burning of coins. + +In addition to the above, the bank module is currently too rigid and handles too many tasks, so this proposal aims to streamline the module by focusing on core functions `Send`, `Mint`, and `Burn`. + +Currently, the module is split across different keepers with scattered and duplicates functionalities (with 4 send functions for instance). + +Additionally, the integration of the token factory into the bank module allows for standardization, and better integration within the core modules. + +This rewrite will reduce complexity and enhance the efficiency and UX of the bank module. + +## Context + +The current implementation of the bank module is characterised by its handling of a broad array of functions, leading to significant complexity in using and extending the bank module. + +These issues have underscored the need for a refactoring strategy that simplifies the module’s architecture and focuses on its most essential operations. + +Additionally, there is an overlap in functionality with a Token Factory module, which could be integrated to streamline oper. + +## Decision + +**Permission Tightening**: Access to the module can be restricted to selected denominations only, ensuring that it operates within designated boundaries and does not exceed its intended scope. Currently, the permissions allow all denoms, so this should be changed. Send restrictions functionality will be maintained. + +**Simplification of Logic**: The bank module will focus on core functionalities `Send`, `Mint`, and `Burn`. This refinement aims to streamline the architecture, enhancing both maintainability and performance. + +**Integration of Token Factory**: The Token Factory will be merged into the bank module. This consolidation of related functionalities aims to reduce redundancy and enhance coherence within the system. Migrations functions will be provided for migrating from Osmosis' Token Factory module to bank/v2. + +**Legacy Support**: A legacy wrapper will be implemented to ensure compatibility with about 90% of existing functions. This measure will facilitate a smooth transition while keeping older systems functional. + +**Denom Implementation**: A asset interface will be added to standardise interactions such as transfers, balance inquiries, minting, and burning across different tokens. This will allow the bank module to support arbitrary asset types, enabling developers to implement custom, ERC20-like denominations. + +For example, currently if a team would like to extend the transfer method the changes would apply universally, affecting all denom’s. With the proposed Asset Interface, it allows teams to customise or extend the transfer method specifically for their own tokens without impacting others. + +These improvements are expected to enhance the flexibility of the bank module, allowing for the creation of custom tokens similar to ERC20 standards and assets backed by CosmWasm (CW) contracts. The integration efforts will also aim to unify CW20 with bank coins across the Cosmos chains. + +Example of denom interface: + +```go +type AssetInterface interface { + Transfer(ctx sdk.Context, from sdk.AccAddress, to sdk.AccAddress, amount sdk.Coin) error + Mint(ctx sdk.Context, to sdk.AccAddress, amount sdk.Coin) error + Burn(ctx sdk.Context, from sdk.AccAddress, amount sdk.Coin) error + QueryBalance(ctx sdk.Context, account sdk.AccAddress) (sdk.Coin, error) +} +``` + +Overview of flow: + +1. Alice initiates a transfer by entering Bob's address and the amount (100 ATOM) +2. The Bank module verifies that the ATOM token implements the `AssetInterface` by querying the `ATOM_Denom_Account`, which is an `x/account` denom account. +3. The Bank module executes the transfer by subtracting 100 ATOM from Alice’s balance and adding 100 ATOM to Bob’s balance. +4. The Bank module calls the Transfer method on the `ATOM_Denom_Account`. The Transfer method, defined in the `AssetInterface`, handles the logic to subtract 100 ATOM from Alice’s balance and add 100 ATOM to Bob’s balance. +5. The Bank module updates the chain and returns the new balances. +6. Both Alice and Bob successfully receive the updated balances. + +## Migration Plans + +Bank is a widely used module, so getting a v2 needs to be thought thoroughly. In order to not force all dependencies to immediately migrate to bank/v2, the same _upgrading_ path will be taken as for the `gov` module. + +This means `cosmossdk.io/bank` will stay one module and there won't be a new `cosmossdk.io/bank/v2` go module. Instead the bank protos will be versioned from `v1beta1` (current bank) to `v2`. + +Bank `v1beta1` endpoints will use the new bank v2 implementation for maximum backward compatibility. + +The bank `v1beta1` keepers will be deprecated and potentially eventually removed, but its proto and messages definitions will remain. + +Additionally, as bank plans to integrate token factory, migrations functions will be provided to migrate from Osmosis token factory implementation (most widely used implementation) to the new bank/v2 token factory. + +## Consequences + +### Positive + +- Simplified interaction with bank APIs +- Backward compatible changes (no contracts or apis broken) +- Optional migration (note: bank `v1beta1` won't get any new feature after bank `v2` release) + +### Neutral + +- Asset implementation not available cross-chain (IBC-ed custom asset should possibly fallback to the default implementation) +- Many assets may slow down bank balances requests + +### Negative + +- Temporarily duplicate functionalities as bank `v1beta1` are `v2` are living alongside +- Difficultity to ever completely remove bank `v1beta1` + +### References + +- Current bank module implementation: https://github.com/cosmos/cosmos-sdk/blob/v0.50.6/x/bank/keeper/keeper.go#L22-L53 +- Osmosis token factory: https://github.com/osmosis-labs/osmosis/tree/v25.0.0/x/tokenfactory/keeper diff --git a/.github/aider/guides/cosmos-sdk.md b/.github/aider/guides/cosmos-sdk.md new file mode 100644 index 0000000..6679af3 --- /dev/null +++ b/.github/aider/guides/cosmos-sdk.md @@ -0,0 +1,685 @@ +# Cosmos SDK Core Components + +## Overview + +The Cosmos SDK is a framework for building secure blockchain applications on CometBFT. It provides: + +- ABCI implementation in Go +- Multi-store persistence layer +- Transaction routing system + +## Transaction Flow + +1. CometBFT consensus delivers transaction bytes +2. SDK decodes transactions and extracts messages +3. Messages routed to appropriate modules +4. State changes committed to stores + +```mermaid +graph TD + A[CometBFT] -->|Tx Bytes| B[SDK Decode] + B -->|Messages| C[Module Router] + C -->|State Changes| D[Multi-store] +``` + +## `baseapp` + +`baseapp` is the boilerplate implementation of a Cosmos SDK application. It comes with an implementation of the ABCI to handle the connection with the underlying consensus engine. Typically, a Cosmos SDK application extends `baseapp` by embedding it in [`app.go`](../beginner/00-app-anatomy.md#core-application-file). + +Here is an example of this from `simapp`, the Cosmos SDK demonstration app: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/simapp/app.go#L145-L186 +``` + +The goal of `baseapp` is to provide a secure interface between the store and the extensible state machine while defining as little about the state machine as possible (staying true to the ABCI). + +For more on `baseapp`, please click [here](../advanced/00-baseapp.md). + +## Multistore + +The Cosmos SDK provides a [`multistore`](../advanced/04-store.md#multistore) for persisting state. The multistore allows developers to declare any number of [`KVStores`](../advanced/04-store.md#base-layer-kvstores). These `KVStores` only accept the `[]byte` type as value and therefore any custom structure needs to be marshalled using [a codec](../advanced/05-encoding.md) before being stored. + +The multistore abstraction is used to divide the state in distinct compartments, each managed by its own module. For more on the multistore, click [here](../advanced/04-store.md#multistore). + +## Modules + +The power of the Cosmos SDK lies in its modularity. Cosmos SDK applications are built by aggregating a collection of interoperable modules. Each module defines a subset of the state and contains its own message/transaction processor, while the Cosmos SDK is responsible for routing each message to its respective module. + +Here is a simplified view of how a transaction is processed by the application of each full-node when it is received in a valid block: + +```mermaid + flowchart TD + A[Transaction relayed from the full-node's CometBFT engine to the node's application via DeliverTx] --> B[APPLICATION] + B -->|"Using baseapp's methods: Decode the Tx, extract and route the message(s)"| C[Message routed to the correct module to be processed] + C --> D1[AUTH MODULE] + C --> D2[BANK MODULE] + C --> D3[STAKING MODULE] + C --> D4[GOV MODULE] + D1 -->|Handle message, Update state| E["Return result to CometBFT (0=Ok, 1=Err)"] + D2 -->|Handle message, Update state| E["Return result to CometBFT (0=Ok, 1=Err)"] + D3 -->|Handle message, Update state| E["Return result to CometBFT (0=Ok, 1=Err)"] + D4 -->|Handle message, Update state| E["Return result to CometBFT (0=Ok, 1=Err)"] +``` + +Each module can be seen as a little state-machine. Developers need to define the subset of the state handled by the module, as well as custom message types that modify the state (_Note:_ `messages` are extracted from `transactions` by `baseapp`). In general, each module declares its own `KVStore` in the `multistore` to persist the subset of the state it defines. Most developers will need to access other 3rd party modules when building their own modules. Given that the Cosmos SDK is an open framework, some of the modules may be malicious, which means there is a need for security principles to reason about inter-module interactions. These principles are based on [object-capabilities](../advanced/10-ocap.md). In practice, this means that instead of having each module keep an access control list for other modules, each module implements special objects called `keepers` that can be passed to other modules to grant a pre-defined set of capabilities. + +Cosmos SDK modules are defined in the `x/` folder of the Cosmos SDK. Some core modules include: + +- `x/auth`: Used to manage accounts and signatures. +- `x/bank`: Used to enable tokens and token transfers. +- `x/staking` + `x/slashing`: Used to build Proof-of-Stake blockchains. + +In addition to the already existing modules in `x/`, which anyone can use in their app, the Cosmos SDK lets you build your own custom modules. You can check an [example of that in the tutorial](https://tutorials.cosmos.network/).# Keepers + +:::note Synopsis +`Keeper`s refer to a Cosmos SDK abstraction whose role is to manage access to the subset of the state defined by various modules. `Keeper`s are module-specific, i.e. the subset of state defined by a module can only be accessed by a `keeper` defined in said module. If a module needs to access the subset of state defined by another module, a reference to the second module's internal `keeper` needs to be passed to the first one. This is done in `app.go` during the instantiation of module keepers. +::: + +:::note Pre-requisite Readings + +- [Introduction to Cosmos SDK Modules](./00-intro.md) + +::: + +## Motivation + +The Cosmos SDK is a framework that makes it easy for developers to build complex decentralized applications from scratch, mainly by composing modules together. As the ecosystem of open-source modules for the Cosmos SDK expands, it will become increasingly likely that some of these modules contain vulnerabilities, as a result of the negligence or malice of their developer. + +The Cosmos SDK adopts an [object-capabilities-based approach](https://docs.cosmos.network/main/learn/advanced/ocap#ocaps-in-practice) to help developers better protect their application from unwanted inter-module interactions, and `keeper`s are at the core of this approach. A `keeper` can be considered quite literally to be the gatekeeper of a module's store(s). Each store (typically an [`IAVL` Store](../../learn/advanced/04-store.md#iavl-store)) defined within a module comes with a `storeKey`, which grants unlimited access to it. The module's `keeper` holds this `storeKey` (which should otherwise remain unexposed), and defines [methods](#implementing-methods) for reading and writing to the store(s). + +The core idea behind the object-capabilities approach is to only reveal what is necessary to get the work done. In practice, this means that instead of handling permissions of modules through access-control lists, module `keeper`s are passed a reference to the specific instance of the other modules' `keeper`s that they need to access (this is done in the [application's constructor function](../../learn/beginner/00-app-anatomy.md#constructor-function)). As a consequence, a module can only interact with the subset of state defined in another module via the methods exposed by the instance of the other module's `keeper`. This is a great way for developers to control the interactions that their own module can have with modules developed by external developers. + +## Type Definition + +`keeper`s are generally implemented in a `/keeper/keeper.go` file located in the module's folder. By convention, the type `keeper` of a module is simply named `Keeper` and usually follows the following structure: + +```go +type Keeper struct { + // External keepers, if any + + // Store key(s) + + // codec + + // authority +} +``` + +For example, here is the type definition of the `keeper` from the `staking` module: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/staking/keeper/keeper.go#L54-L115 +``` + +Let us go through the different parameters: + +- An expected `keeper` is a `keeper` external to a module that is required by the internal `keeper` of said module. External `keeper`s are listed in the internal `keeper`'s type definition as interfaces. These interfaces are themselves defined in an `expected_keepers.go` file in the root of the module's folder. In this context, interfaces are used to reduce the number of dependencies, as well as to facilitate the maintenance of the module itself. +- `KVStoreService`s grant access to the store(s) of the [multistore](../../learn/advanced/04-store.md) managed by the module. They should always remain unexposed to external modules. +- `cdc` is the [codec](../../learn/advanced/05-encoding.md) used to marshal and unmarshal structs to/from `[]byte`. The `cdc` can be any of `codec.BinaryCodec`, `codec.JSONCodec` or `codec.Codec` based on your requirements. It can be either a proto or amino codec as long as they implement these interfaces. +- The authority listed is a module account or user account that has the right to change module level parameters. Previously this was handled by the param module, which has been deprecated. + +Of course, it is possible to define different types of internal `keeper`s for the same module (e.g. a read-only `keeper`). Each type of `keeper` comes with its own constructor function, which is called from the [application's constructor function](../../learn/beginner/00-app-anatomy.md). This is where `keeper`s are instantiated, and where developers make sure to pass correct instances of modules' `keeper`s to other modules that require them. + +## Implementing Methods + +`Keeper`s primarily expose methods for business logic, as validity checks should have already been performed by the [`Msg` server](./03-msg-services.md) when `keeper`s' methods are called. + + + +State management is recommended to be done via [Collections](../packages/collections) + + + +## State Management + +In the Cosmos SDK, it is crucial to be methodical and selective when managing state within a module, as improper state management can lead to inefficiency, security risks, and scalability issues. Not all data belongs in the on-chain state; it's important to store only essential blockchain data that needs to be verified by consensus. Storing unnecessary information, especially client-side data, can bloat the state and slow down performance. Instead, developers should focus on using an off-chain database to handle supplementary data, extending the API as needed. This approach minimizes on-chain complexity, optimizes resource usage, and keeps the blockchain state lean and efficient, ensuring scalability and smooth operations. + +The Cosmos SDK leverages Protocol Buffers (protobuf) for efficient state management, providing a well-structured, binary encoding format that ensures compatibility and performance across different modules. The SDK’s recommended approach for managing state is through the [collections package](../pacakges/02-collections.md), which simplifies state handling by offering predefined data structures like maps and indexed sets, reducing the complexity of managing raw state data. While users can opt for custom encoding schemes if they need more flexibility or have specialized requirements, they should be aware that such custom implementations may not integrate seamlessly with indexers that decode state data on the fly. This could lead to challenges in data retrieval, querying, and interoperability, making protobuf a safer and more future-proof choice for most use cases. + +# Folder Structure + +:::note Synopsis +This document outlines the structure of Cosmos SDK modules. These ideas are meant to be applied as suggestions. Application developers are encouraged to improve upon and contribute to module structure and development design. + +The required interface for a module is located in the module.go. Everything beyond this is suggestive. +::: + +## Structure + +A typical Cosmos SDK module can be structured as follows: + +```shell +proto +└── {project_name} +    └── {module_name} +    └── {proto_version} +       ├── {module_name}.proto +       ├── genesis.proto +       ├── query.proto +       └── tx.proto +``` + +- `{module_name}.proto`: The module's common message type definitions. +- `genesis.proto`: The module's message type definitions related to genesis state. +- `query.proto`: The module's Query service and related message type definitions. +- `tx.proto`: The module's Msg service and related message type definitions. + +```shell +x/{module_name} +├── client +│   ├── cli +│   │ ├── query.go +│   │   └── tx.go +│   └── testutil +│   ├── cli_test.go +│   └── suite.go +├── exported +│   └── exported.go +├── keeper +│   ├── genesis.go +│   ├── grpc_query.go +│   ├── hooks.go +│   ├── invariants.go +│   ├── keeper.go +│   ├── keys.go +│   ├── msg_server.go +│   └── querier.go +├── simulation +│   ├── decoder.go +│   ├── genesis.go +│   ├── operations.go +│   └── params.go +├── types +│   ├── {module_name}.pb.go +│ ├── codec.go +│ ├── errors.go +│ ├── events.go +│ ├── events.pb.go +│ ├── expected_keepers.go +│ ├── genesis.go +│ ├── genesis.pb.go +│ ├── keys.go +│ ├── msgs.go +│ ├── params.go +│ ├── query.pb.go +│ └── tx.pb.go +├── module.go +├── abci.go +├── autocli.go +├── depinject.go +└── README.md +``` + +- `client/`: The module's CLI client functionality implementation and the module's CLI testing suite. +- `exported/`: The module's exported types - typically interface types. If a module relies on keepers from another module, it is expected to receive the keepers as interface contracts through the `expected_keepers.go` file (see below) in order to avoid a direct dependency on the module implementing the keepers. However, these interface contracts can define methods that operate on and/or return types that are specific to the module that is implementing the keepers and this is where `exported/` comes into play. The interface types that are defined in `exported/` use canonical types, allowing for the module to receive the keepers as interface contracts through the `expected_keepers.go` file. This pattern allows for code to remain DRY and also alleviates import cycle chaos. +- `keeper/`: The module's `Keeper` and `MsgServer` implementation. + - `abci.go`: The module's `BeginBlocker` and `EndBlocker` implementations (this file is only required if `BeginBlocker` and/or `EndBlocker` need to be defined). +- `simulation/`: The module's [simulation](./14-simulator.md) package defines functions used by the blockchain simulator application (`simapp`). +- `README.md`: The module's specification documents outlining important concepts, state storage structure, and message and event type definitions. Learn more how to write module specs in the [spec guidelines](../spec/SPEC_MODULE.md). +- `types/`: includes type definitions for messages, events, and genesis state, including the type definitions generated by Protocol Buffers. + - `codec.go`: The module's registry methods for interface types. + - `errors.go`: The module's sentinel errors. + - `events.go`: The module's event types and constructors. + - `expected_keepers.go`: The module's [expected keeper](./06-keeper.md#type-definition) interfaces. + - `genesis.go`: The module's genesis state methods and helper functions. + - `keys.go`: The module's store keys and associated helper functions. + - `msgs.go`: The module's message type definitions and associated methods. + - `params.go`: The module's parameter type definitions and associated methods. + - `*.pb.go`: The module's type definitions generated by Protocol Buffers (as defined in the respective `*.proto` files above). +- The root directory includes the module's `AppModule` implementation. + - `autocli.go`: The module [autocli](https://docs.cosmos.network/main/core/autocli) options. + - `depinject.go`: The module [depinject](./15-depinject.md#type-definition) options. + +> Note: although the above pattern is followed by most of the Cosmos SDK modules, there are some modules that don't follow this pattern. E.g `x/group` and `x/nft` dont have a `types` folder, instead all of the type definitions for messages, events, and genesis state are live in the root directory and the module's `AppModule` implementation lives in the `module` folder. + +--- + +## sidebar_position: 1 + +# `Msg` Services + +:::note Synopsis +A Protobuf `Msg` service processes [messages](./02-messages-and-queries.md#messages). Protobuf `Msg` services are specific to the module in which they are defined, and only process messages defined within the said module. They are called from `BaseApp` during [`FinalizeBlock`](../../learn/advanced/00-baseapp.md#finalizeblock). +::: + +:::note Pre-requisite Readings + +- [Module Manager](./01-module-manager.md) +- [Messages and Queries](./02-messages-and-queries.md) + +::: + +## Implementation of a module `Msg` service + +Each module should define a Protobuf `Msg` service, which will be responsible for processing requests (implementing `sdk.Msg`) and returning responses. + +As further described in [ADR 031](../architecture/adr-031-msg-service.md), this approach has the advantage of clearly specifying return types and generating server and client code. + +Protobuf generates a `MsgServer` interface based on the definition of `Msg` service. It is the role of the module developer to implement this interface, by implementing the state transition logic that should happen upon receival of each `transaction.Msg`. As an example, here is the generated `MsgServer` interface for `x/bank`, which exposes two `transaction.Msg`s: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/28fa3b8/x/bank/types/tx.pb.go#L564-L579 +``` + +When possible, the existing module's [`Keeper`](./06-keeper.md) should implement `MsgServer`, otherwise a `msgServer` struct that embeds the `Keeper` can be created, typically in `./keeper/msg_server.go`: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/28fa3b8/x/bank/keeper/msg_server.go#L16-L19 +``` + +`msgServer` methods can retrieve the auxiliary information or services using the environment variable, it is always located in the keeper: + +Environment: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/07151304e2ec6a185243d083f59a2d543253cb15/core/appmodule/v2/environment.go#L14-L29 +``` + +Keeper Example: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/07151304e2ec6a185243d083f59a2d543253cb15/x/bank/keeper/keeper.go#L56-L58 +``` + +`transaction.Msg` processing usually follows these 3 steps: + +### Validation + +The message server must perform all validation required (both _stateful_ and _stateless_) to make sure the `message` is valid. +The `signer` is charged for the gas cost of this validation. + +For example, a `msgServer` method for a `transfer` message should check that the sending account has enough funds to actually perform the transfer. + +It is recommended to implement all validation checks in a separate function that passes state values as arguments. This implementation simplifies testing. As expected, expensive validation functions charge additional gas. Example: + +```go +ValidateMsgA(msg MsgA, now Time, gm GasMeter) error { + if now.Before(msg.Expire) { + return sdkerrors.ErrInvalidRequest.Wrap("msg expired") + } + gm.ConsumeGas(1000, "signature verification") + return signatureVerificaton(msg.Prover, msg.Data) +} +``` + +:::warning +Previously, the `ValidateBasic` method was used to perform simple and stateless validation checks. +This way of validating is deprecated, this means the `msgServer` must perform all validation checks. +::: + +### State Transition + +After the validation is successful, the `msgServer` method uses the [`keeper`](./06-keeper.md) functions to access the state and perform a state transition. + +### Events + +Before returning, `msgServer` methods generally emit one or more [events](../../learn/advanced/08-events.md) by using the `EventManager` held in `environment`. + +There are two ways to emit events, typed events using protobuf or arbitrary key & values. + +Typed Events: + +```go +ctx.EventManager().EmitTypedEvent( + &group.EventABC{Key1: Value1, Key2, Value2}) +``` + +Arbitrary Events: + +```go +ctx.EventManager().EmitEvent( + sdk.NewEvent( + eventType, // e.g. sdk.EventTypeMessage for a message, types.CustomEventType for a custom event defined in the module + sdk.NewAttribute(key1, value1), + sdk.NewAttribute(key2, value2), + ), +) +``` + +These events are relayed back to the underlying consensus engine and can be used by service providers to implement services around the application. Click [here](../../learn/advanced/08-events.md) to learn more about events. + +The invoked `msgServer` method returns a `proto.Message` response and an `error`. These return values are then wrapped into an `*sdk.Result` or an `error`: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/baseapp/msg_service_router.go#L160 +``` + +This method takes care of marshaling the `res` parameter to protobuf and attaching any events on the `EventManager()` to the `sdk.Result`. + +```protobuf reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/proto/cosmos/base/abci/v1beta1/abci.proto#L93-L113 +``` + +This diagram shows a typical structure of a Protobuf `Msg` service, and how the message propagates through the module. + +```mermaid +sequenceDiagram + participant User + participant baseApp + participant router + participant handler + participant msgServer + participant keeper + participant EventManager + + User->>baseApp: Transaction Type + baseApp->>router: Route(ctx, msgRoute) + router->>handler: handler + handler->>msgServer: Msg(Context, Msg(..)) + + alt addresses invalid, denominations wrong, etc. + msgServer->>handler: error + handler->>router: error + router->>baseApp: result, error code + else + msgServer->>keeper: perform action, update context + keeper->>msgServer: results, error code + msgServer->>EventManager: Emit relevant events + msgServer->>msgServer: maybe wrap results in more structure + msgServer->>handler: result, error code + handler->>router: result, error code + router->>baseApp: result, error code + end + + baseApp->>User: result, error code +``` + +## Telemetry + +New [telemetry metrics](../../learn/advanced/09-telemetry.md) can be created from `msgServer` methods when handling messages. + +This is an example from the `x/auth/vesting` module: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/auth/vesting/msg_server.go#L76-L88 +``` + +:::Warning +Telemetry adds a performance overhead to the chain. It is recommended to only use this in critical paths +::: + +--- + +## sidebar_position: 1 + +# Query Services + +:::note Synopsis +A Protobuf Query service processes [`queries`](./02-messages-and-queries.md#queries). Query services are specific to the module in which they are defined, and only process `queries` defined within said module. They are called from `BaseApp`'s [`Query` method](../../learn/advanced/00-baseapp.md#query). +::: + +:::note Pre-requisite Readings + +- [Module Manager](./01-module-manager.md) +- [Messages and Queries](./02-messages-and-queries.md) + +::: + +## Implementation of a module query service + +### gRPC Service + +When defining a Protobuf `Query` service, a `QueryServer` interface is generated for each module with all the service methods: + +```go +type QueryServer interface { + QueryBalance(context.Context, *QueryBalanceParams) (*types.Coin, error) + QueryAllBalances(context.Context, *QueryAllBalancesParams) (*QueryAllBalancesResponse, error) +} +``` + +These custom queries methods should be implemented by a module's keeper, typically in `./keeper/grpc_query.go`. The first parameter of these methods is a generic `context.Context`. Therefore, the Cosmos SDK provides a function `sdk.UnwrapSDKContext` to retrieve the `context.Context` from the provided +`context.Context`. + +Here's an example implementation for the bank module: + +```go reference +https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/bank/keeper/grpc_query.go +``` + +### Calling queries from the State Machine + +The Cosmos SDK v0.47 introduces a new `cosmos.query.v1.module_query_safe` Protobuf annotation which is used to state that a query that is safe to be called from within the state machine, for example: + +- a Keeper's query function can be called from another module's Keeper, +- ADR-033 intermodule query calls, +- CosmWasm contracts can also directly interact with these queries. + +If the `module_query_safe` annotation set to `true`, it means: + +- The query is deterministic: given a block height it will return the same response upon multiple calls, and doesn't introduce any state-machine breaking changes across SDK patch versions. +- Gas consumption never fluctuates across calls and across patch versions. + +If you are a module developer and want to use `module_query_safe` annotation for your own query, you have to ensure the following things: + +- the query is deterministic and won't introduce state-machine-breaking changes without coordinated upgrades +- it has its gas tracked, to avoid the attack vector where no gas is accounted for + on potentially high-computation queries. + + *** + + sidebar_position: 1 + +--- + +# Blockchain Architecture + +## Introduction + +Blockchain architecture is a complex topic that involves many different components. In this section, we will cover the main layers of a blockchain application built with the Cosmos SDK. + +At its core, a blockchain is a replicated deterministic state machine. This document explores the various layers of blockchain architecture, focusing on the execution, settlement, consensus, data availability, and interoperability layers. + +```mermaid +graph TD + A[Modular SDK Blockchain Architecture] + A --> B[Execution Layer] + A --> C[Settlement Layer] + A --> D[Consensus Layer] + D --> E[Data Availability Layer] + A --> F[Interoperability Layer] +``` + +## Layered Architecture + +Understanding blockchain architecture through the lens of different layers helps in comprehending its complex functionalities. We will give a high-level overview of the execution layer, settlement layer, consensus layer, data availability layer, and interoperability layer. + +## Execution Layer + +The Execution Layer is where the blockchain processes and executes transactions. The state machine within the blockchain handles the execution of transaction logic. This is done by the blockchain itself, ensuring that every transaction follows the predefined rules and state transitions. When a transaction is submitted, the execution layer processes it, updates the state, and ensures that the output is deterministic and consistent across all nodes. In the context of the Cosmos SDK, this typically involves predefined modules and transaction types rather than general-purpose smart contracts, which are used in chains with CosmWasm. + +### State machine + +At its core, a blockchain is a [replicated deterministic state machine](https://en.wikipedia.org/wiki/State_machine_replication). + +A state machine is a computer science concept whereby a machine can have multiple states, but only one at any given time. There is a `state`, which describes the current state of the system, and `transactions`, that trigger state transitions. + +Given a state S and a transaction T, the state machine will return a new state S'. + +```mermaid +flowchart LR + A[S] + B[S'] + A -->|"apply(T)"| B +``` + +In practice, the transactions are bundled in blocks to make the process more efficient. Given a state S and a block of transactions B, the state machine will return a new state S'. + +```mermaid +flowchart LR + A[S] + B[S'] + A -->|"For each T in B: apply(T)"| B +``` + +In a blockchain context, the state machine is [deterministic](https://en.wikipedia.org/wiki/Deterministic_system). This means that if a node is started at a given state and replays the same sequence of transactions, it will always end up with the same final state. + +The Cosmos SDK gives developers maximum flexibility to define the state of their application, transaction types and state transition functions. The process of building state machines with the Cosmos SDK will be described more in-depth in the following sections. But first, let us see how the state machine is replicated using various consensus engines, such as CometBFT. + +## Settlement Layer + +The Settlement Layer is responsible for finalising and recording transactions on the blockchain. This layer ensures that all transactions are accurately settled and immutable, providing a verifiable record of all activities on the blockchain. It is critical for maintaining the integrity and trustworthiness of the blockchain. + +The settlement layer can be performed on the chain itself or it can be externalised, allowing for the possibility of plugging in a different settlement layer as needed. For example if we were to use Rollkit and celestia for our Data Availability and Consensus, we could separate our settlement layer by introducing fraud or validity proofs. From there the settlement layer can create trust-minimised light clients, further enhancing security and efficiency. This process ensures that all transactions are accurately finalized and immutable, providing a verifiable record of all activities. + +## Consensus Layer + +The Consensus Layer ensures that all nodes in the network agree on the order and validity of transactions. This layer uses consensus algorithms like Byzantine Fault Tolerance (BFT) or Proof of Stake (PoS) to achieve agreement, even in the presence of malicious nodes. Consensus is crucial for maintaining the security and reliability of the blockchain. + +What has been a default consensus engine in the Cosmos SDK has been CometBFT. In the most recent releases we have been moving away from this and allowing users to plug and play their own consensus engines. This is a big step forward for the Cosmos SDK as it allows for more flexibility and customisation. Other consensus engine options for example can be Rollkit with Celestias Data Availability Layer. + +Here is an example of how the consensus layer works with CometBFT in the context of the Cosmos SDK: + +### CometBFT + +Thanks to the Cosmos SDK, developers just have to define the state machine, and [_CometBFT_](https://docs.cometbft.com/v1.0/explanation/introduction/) will handle replication over the network for them. + +```mermaid +flowchart TD + subgraph Blockchain_Node[Blockchain Node] + subgraph SM[State-machine] + direction TB + SM1[Cosmos SDK] + end + subgraph CometBFT[CometBFT] + direction TB + Consensus + Networking + end + end + + SM <--> CometBFT + + + Blockchain_Node -->|Includes| SM + Blockchain_Node -->|Includes| CometBFT +``` + +[CometBFT](https://docs.cometbft.com/v1.0/explanation/introduction/) is an application-agnostic engine that is responsible for handling the _networking_ and _consensus_ layers of a blockchain. In practice, this means that CometBFT is responsible for propagating and ordering transaction bytes. CometBFT relies on an eponymous Byzantine-Fault-Tolerant (BFT) algorithm to reach consensus on the order of transactions. + +The [consensus algorithm adopted by CometBFT](https://docs.cometbft.com/v1.0/explanation/introduction/#consensus-overview) works with a set of special nodes called _Validators_. Validators are responsible for adding blocks of transactions to the blockchain. At any given block, there is a validator set V. A validator in V is chosen by the algorithm to be the proposer of the next block. This block is considered valid if more than two thirds of V signed a `prevote` and a `precommit` on it, and if all the transactions that it contains are valid. The validator set can be changed by rules written in the state-machine. + +## ABCI + +CometBFT passes transactions to the application through an interface called the [ABCI](https://docs.cometbft.com/v1.0/spec/abci/), which the application must implement. + +```mermaid +graph TD + A[Application] + B[CometBFT] + A <-->|ABCI| B + +``` + +Note that **CometBFT only handles transaction bytes**. It has no knowledge of what these bytes mean. All CometBFT does is order these transaction bytes deterministically. CometBFT passes the bytes to the application via the ABCI, and expects a return code to inform it if the messages contained in the transactions were successfully processed or not. + +Here are the most important messages of the ABCI: + +- `CheckTx`: When a transaction is received by CometBFT, it is passed to the application to check if a few basic requirements are met. `CheckTx` is used to protect the mempool of full-nodes against spam transactions. A special handler called the [`AnteHandler`](../beginner/04-gas-fees.md#antehandler) is used to execute a series of validation steps such as checking for sufficient fees and validating the signatures. If the checks are valid, the transaction is added to the [mempool](https://docs.cometbft.com/v1.0/explanation/core/mempool) and relayed to peer nodes. Note that transactions are not processed (i.e. no modification of the state occurs) with `CheckTx` since they have not been included in a block yet. +- `DeliverTx`: When a [valid block](https://docs.cometbft.com/v1.0/spec/core/data_structures#block) is received by CometBFT, each transaction in the block is passed to the application via `DeliverTx` in order to be processed. It is during this stage that the state transitions occur. The `AnteHandler` executes again, along with the actual [`Msg` service](../../build/building-modules/03-msg-services.md) RPC for each message in the transaction. +- `BeginBlock`/`EndBlock`: These messages are executed at the beginning and the end of each block, whether the block contains transactions or not. It is useful to trigger automatic execution of logic. Proceed with caution though, as computationally expensive loops could slow down your blockchain, or even freeze it if the loop is infinite. + +Find a more detailed view of the ABCI methods from the [CometBFT docs](https://docs.cometbft.com/v1.0/spec/abci/). + +Any application built on CometBFT needs to implement the ABCI interface in order to communicate with the underlying local CometBFT engine. Fortunately, you do not have to implement the ABCI interface. The Cosmos SDK provides a boilerplate implementation of it in the form of [baseapp](./03-sdk-design.md#baseapp). + +## Data Availability Layer + +The Data Availability (DA) Layer is a critical component of within the umbrella of the consensus layer that ensures all necessary data for transactions is available to all network participants. This layer is essential for preventing data withholding attacks, where some nodes might attempt to disrupt the network by not sharing critical transaction data. + +If we use the example of Rollkit, a user initiates a transaction, which is then propagated through the rollup network by a light node. The transaction is validated by full nodes and aggregated into a block by the sequencer. This block is posted to a data availability layer like Celestia, ensuring the data is accessible and correctly ordered. The rollup light node verifies data availability from the DA layer. Full nodes then validate the block and generate necessary proofs, such as fraud proofs for optimistic rollups or zk-SNARKs/zk-STARKs for zk-rollups. These proofs are shared across the network and verified by other nodes, ensuring the rollup's integrity. Once all validations are complete, the rollup's state is updated, finalising the transaction + +## Interoperability Layer + +The Interoperability Layer enables communication and interaction between different blockchains. This layer facilitates cross-chain transactions and data sharing, allowing various blockchain networks to interoperate seamlessly. Interoperability is key for building a connected ecosystem of blockchains, enhancing their functionality and reach. + +In this case we have separated the layers even further to really illustrate the components that make-up the blockchain architecture and it is important to note that the Cosmos SDK is designed to be interoperable with other blockchains. This is achieved through the use of the [Inter-Blockchain Communication (IBC) protocol](https://www.ibcprotocol.dev/), which allows different blockchains to communicate and transfer assets between each other. + +--- + +## sidebar_position: 1 + +# Application-Specific Blockchains + +:::note Synopsis +This document explains what application-specific blockchains are, and why developers would want to build one as opposed to writing Smart Contracts. +::: + +## What are application-specific blockchains + +Application-specific blockchains are blockchains customized to operate a single application. Instead of building a decentralized application on top of an underlying blockchain like Ethereum, developers build their own blockchain from the ground up. This means building a full-node client, a light-client, and all the necessary interfaces (CLI, REST, ...) to interact with the nodes. + +```mermaid +flowchart TD + subgraph Blockchain_Node[Blockchain Node] + subgraph SM[State-machine] + direction TB + SM1[Cosmos SDK] + end + subgraph Consensus[Consensus] + direction TB + end + subgraph Networking[Networking] + direction TB + end + end + + SM <--> Consensus + Consensus <--> Networking + + + Blockchain_Node -->|Includes| SM + Blockchain_Node -->|Includes| Consensus + Blockchain_Node -->|Includes| Networking +``` + +## What are the shortcomings of Smart Contracts + +Virtual-machine blockchains like Ethereum addressed the demand for more programmability back in 2014. At the time, the options available for building decentralized applications were quite limited. Most developers would build on top of the complex and limited Bitcoin scripting language, or fork the Bitcoin codebase which was hard to work with and customize. + +Virtual-machine blockchains came in with a new value proposition. Their state-machine incorporates a virtual-machine that is able to interpret turing-complete programs called Smart Contracts. These Smart Contracts are very good for use cases like one-time events (e.g. ICOs), but they can fall short for building complex decentralized platforms. Here is why: + +- Smart Contracts are generally developed with specific programming languages that can be interpreted by the underlying virtual-machine. These programming languages are often immature and inherently limited by the constraints of the virtual-machine itself. For example, the Ethereum Virtual Machine does not allow developers to implement automatic execution of code. Developers are also limited to the account-based system of the EVM, and they can only choose from a limited set of functions for their cryptographic operations. These are examples, but they hint at the lack of **flexibility** that a smart contract environment often entails. +- Smart Contracts are all run by the same virtual machine. This means that they compete for resources, which can severely restrain **performance**. And even if the state-machine were to be split in multiple subsets (e.g. via sharding), Smart Contracts would still need to be interpreted by a virtual machine, which would limit performance compared to a native application implemented at state-machine level (our benchmarks show an improvement on the order of 10x in performance when the virtual-machine is removed). +- Another issue with the fact that Smart Contracts share the same underlying environment is the resulting limitation in **sovereignty**. A decentralized application is an ecosystem that involves multiple players. If the application is built on a general-purpose virtual-machine blockchain, stakeholders have very limited sovereignty over their application, and are ultimately superseded by the governance of the underlying blockchain. If there is a bug in the application, very little can be done about it. + +Application-Specific Blockchains are designed to address these shortcomings. + +## Application-Specific Blockchains Benefits + +### Flexibility + +Application-specific blockchains give maximum flexibility to developers: + +- In Cosmos blockchains, the state-machine is typically connected to the underlying consensus engine via an interface called the [ABCI](https://docs.cometbft.com/v1.0/spec/abci/) (Application Blockchain Interface). This interface can be wrapped in any programming language, meaning developers can build their state-machine in the programming language of their choice. + +- Developers can choose among multiple frameworks to build their state-machine. The most widely used today is the Cosmos SDK, but others exist (e.g. [Lotion](https://github.com/nomic-io/lotion), [Weave](https://github.com/iov-one/weave), ...). Typically the choice will be made based on the programming language they want to use (Cosmos SDK and Weave are in Golang, Lotion is in Javascript, ...). +- The ABCI also allows developers to swap the consensus engine of their application-specific blockchain. Today, only CometBFT is production-ready, but in the future other consensus engines are expected to emerge. +- Even when they settle for a framework and consensus engine, developers still have the freedom to tweak them if they don't perfectly match their requirements in their pristine forms. +- Developers are free to explore the full spectrum of tradeoffs (e.g. number of validators vs transaction throughput, safety vs availability in asynchrony, ...) and design choices (DB or IAVL tree for storage, UTXO or account model, ...). +- Developers can implement automatic execution of code. In the Cosmos SDK, logic can be automatically triggered at the beginning and the end of each block. They are also free to choose the cryptographic library used in their application, as opposed to being constrained by what is made available by the underlying environment in the case of virtual-machine blockchains. + +The list above contains a few examples that show how much flexibility application-specific blockchains give to developers. The goal of Cosmos and the Cosmos SDK is to make developer tooling as generic and composable as possible, so that each part of the stack can be forked, tweaked and improved without losing compatibility. As the community grows, more alternatives for each of the core building blocks will emerge, giving more options to developers. + +### Performance + +Decentralized applications built with Smart Contracts are inherently capped in performance by the underlying environment. For a decentralized application to optimise performance, it needs to be built as an application-specific blockchain. Next are some of the benefits an application-specific blockchain brings in terms of performance: + +- Developers of application-specific blockchains can choose to operate with a novel consensus engine such as CometBFT. +- An application-specific blockchain only operates a single application, so that the application does not compete with others for computation and storage. This is the opposite of most non-sharded virtual-machine blockchains today, where smart contracts all compete for computation and storage. +- Even if a virtual-machine blockchain offered application-based sharding coupled with an efficient consensus algorithm, performance would still be limited by the virtual-machine itself. The real throughput bottleneck is the state-machine, and requiring transactions to be interpreted by a virtual-machine significantly increases the computational complexity of processing them. + +### Security + +Security is hard to quantify, and greatly varies from platform to platform. That said here are some important benefits an application-specific blockchain can bring in terms of security: + +- Developers can choose proven programming languages like Go when building their application-specific blockchains, as opposed to smart contract programming languages that are often more immature. +- Developers are not constrained by the cryptographic functions made available by the underlying virtual-machines. They can use their own custom cryptography, and rely on well-audited crypto libraries. +- Developers do not have to worry about potential bugs or exploitable mechanisms in the underlying virtual-machine, making it easier to reason about the security of the application. + +### Sovereignty + +One of the major benefits of application-specific blockchains is sovereignty. A decentralized application is an ecosystem that involves many actors: users, developers, third-party services, and more. When developers build on virtual-machine blockchain where many decentralized applications coexist, the community of the application is different than the community of the underlying blockchain, and the latter supersedes the former in the governance process. If there is a bug or if a new feature is needed, stakeholders of the application have very little leeway to upgrade the code. If the community of the underlying blockchain refuses to act, nothing can happen. + +The fundamental issue here is that the governance of the application and the governance of the network are not aligned. This issue is solved by application-specific blockchains. Because application-specific blockchains specialize to operate a single application, stakeholders of the application have full control over the entire chain. This ensures that the community will not be stuck if a bug is discovered, and that it has the freedom to choose how it is going to evolve. diff --git a/.github/aider/guides/sonr-did.md b/.github/aider/guides/sonr-did.md new file mode 100644 index 0000000..e687d0d --- /dev/null +++ b/.github/aider/guides/sonr-did.md @@ -0,0 +1,141 @@ +# `x/did` + +The Decentralized Identity module is responsible for managing native Sonr Accounts, their derived wallets, and associated user identification information. + +## State + +The DID module maintains several key state structures: + +### Controller State + +The Controller state represents a Sonr DWN Vault. It includes: +- Unique identifier (number) +- DID +- Sonr address +- Ethereum address +- Bitcoin address +- Public key +- Keyshares pointer +- Claimed block +- Creation block + +### Assertion State + +The Assertion state includes: +- DID +- Controller +- Subject +- Public key +- Assertion type +- Accumulator (metadata) +- Creation block + +### Authentication State + +The Authentication state includes: +- DID +- Controller +- Subject +- Public key +- Credential ID +- Metadata +- Creation block + +### Verification State + +The Verification state includes: +- DID +- Controller +- DID method +- Issuer +- Subject +- Public key +- Verification type +- Metadata +- Creation block + +## State Transitions + +State transitions are triggered by the following messages: +- LinkAssertion +- LinkAuthentication +- UnlinkAssertion +- UnlinkAuthentication +- ExecuteTx +- UpdateParams + +## Messages + +The DID module defines the following messages: + +1. MsgLinkAuthentication +2. MsgLinkAssertion +3. MsgExecuteTx +4. MsgUnlinkAssertion +5. MsgUnlinkAuthentication +6. MsgUpdateParams + +Each message triggers specific state machine behaviors related to managing DIDs, authentications, assertions, and module parameters. + +## Query + +The DID module provides the following query endpoints: + +1. Params: Query all parameters of the module +2. Resolve: Query the DID document by its ID +3. Sign: Sign a message with the DID document +4. Verify: Verify a message with the DID document + +## Params + +The module parameters include: +- Allowed public keys (map of KeyInfo) +- Conveyance preference +- Attestation formats + +## Client + +The module provides gRPC and REST endpoints for all defined messages and queries. + +## Future Improvements + +Potential future improvements could include: +1. Enhanced privacy features for DID operations +2. Integration with more blockchain networks +3. Support for additional key types and cryptographic algorithms +4. Improved revocation mechanisms for credentials and assertions + +## Tests + +Acceptance tests should cover all major functionality, including: +- Creating and managing DIDs +- Linking and unlinking assertions and authentications +- Executing transactions with DIDs +- Querying and resolving DIDs +- Parameter updates + +## Appendix + +### Account + +An Account represents a user's identity within the Sonr ecosystem. It includes information such as the user's public key, associated wallets, and other identification details. + +### Decentralized Identifier (DID) + +A Decentralized Identifier (DID) is a unique identifier that is created, owned, and controlled by the user. It is used to establish a secure and verifiable digital identity. + +### Verifiable Credential (VC) + +A Verifiable Credential (VC) is a digital statement that can be cryptographically verified. It contains claims about a subject (e.g., a user) and is issued by a trusted authority. + +### Key Types + +The module supports various key types, including: +- Role +- Algorithm (e.g., ES256, EdDSA, ES256K) +- Encoding (e.g., hex, base64, multibase) +- Curve (e.g., P256, P384, P521, X25519, X448, Ed25519, Ed448, secp256k1) + +### JSON Web Key (JWK) + +The module supports JSON Web Keys (JWK) for representing cryptographic keys, including properties such as key type (kty), curve (crv), and coordinates (x, y) for EC and OKP keys, as well as modulus (n) and exponent (e) for RSA keys. diff --git a/.github/aider/guides/sonr-dwn.md b/.github/aider/guides/sonr-dwn.md new file mode 100644 index 0000000..ecca8c1 --- /dev/null +++ b/.github/aider/guides/sonr-dwn.md @@ -0,0 +1,145 @@ +# `x/dwn` + +The DWN module is responsible for the management of IPFS deployed Decentralized Web Nodes (DWNs) and their associated data. + +## Concepts + +The DWN module introduces several key concepts: + +1. Decentralized Web Node (DWN): A distributed network for storing and sharing data. +2. Schema: A structure defining the format of various data types in the dwn. +3. IPFS Integration: The module can interact with IPFS for decentralized data storage. + +## State + +The DWN module maintains the following state: + +### DWN State + +The DWN state is stored using the following structure: + +```protobuf +message DWN { + uint64 id = 1; + string alias = 2; + string cid = 3; + string resolver = 4; +} +``` + +This state is indexed by ID, alias, and CID for efficient querying. + +### Params State + +The module parameters are stored in the following structure: + +```protobuf +message Params { + bool ipfs_active = 1; + bool local_registration_enabled = 2; + Schema schema = 4; +} +``` + +### Schema State + +The Schema state defines the structure for various data types: + +```protobuf +message Schema { + int32 version = 1; + string account = 2; + string asset = 3; + string chain = 4; + string credential = 5; + string did = 6; + string jwk = 7; + string grant = 8; + string keyshare = 9; + string profile = 10; +} +``` + +## State Transitions + +State transitions in the DWN module are primarily triggered by: + +1. Updating module parameters +2. Allocating new dwns +3. Syncing DID documents + +## Messages + +The DWN module defines the following message: + +1. `MsgUpdateParams`: Used to update the module parameters. + +```protobuf +message MsgUpdateParams { + string authority = 1; + Params params = 2; +} +``` + +## Begin Block + +No specific begin-block operations are defined for this module. + +## End Block + +No specific end-block operations are defined for this module. + +## Hooks + +The DWN module does not define any hooks. + +## Events + +The DWN module does not explicitly define any events. However, standard Cosmos SDK events may be emitted during state transitions. + +## Client + +The DWN module provides the following gRPC query endpoints: + +1. `Params`: Queries all parameters of the module. +2. `Schema`: Queries the DID document schema. +3. `Allocate`: Initializes a Target DWN available for claims. +4. `Sync`: Queries the DID document by its ID and returns required information. + +## Params + +The module parameters include: + +- `ipfs_active` (bool): Indicates if IPFS integration is active. +- `local_registration_enabled` (bool): Indicates if local registration is enabled. +- `schema` (Schema): Defines the structure for various data types in the dwn. + +## Future Improvements + +Potential future improvements could include: + +1. Enhanced IPFS integration features. +2. Additional authentication mechanisms beyond WebAuthn. +3. Improved DID document management and querying capabilities. + +## Tests + +Acceptance tests should cover: + +1. Parameter updates +2. DWN state management +3. Schema queries +4. DWN allocation process +5. DID document syncing + +## Appendix + +| Concept | Description | +| ------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Decentralized Web Node (DWN) | A decentralized, distributed, and secure network of nodes that store and share data. It is a decentralized alternative to traditional web hosting services. | +| Decentralized Identifier (DID) | A unique identifier that is created, owned, and controlled by the user. It is used to establish a secure and verifiable digital identity. | +| HTMX (Hypertext Markup Language eXtensions) | A set of extensions to HTML that allow for the creation of interactive web pages. It is used to enhance the user experience and provide additional functionality to web applications. | +| IPFS (InterPlanetary File System) | A decentralized, peer-to-peer network for storing and sharing data. It is a distributed file system that allows for the creation and sharing of content across a network of nodes. | +| WebAuthn (Web Authentication) | A set of APIs that allow websites to request user authentication using biometric or non-biometric factors. | +| WebAssembly (Web Assembly) | A binary instruction format for a stack-based virtual machine. | +| Verifiable Credential (VC) | A digital statement that can be cryptographically verified. | diff --git a/.github/aider/guides/sonr-service.md b/.github/aider/guides/sonr-service.md new file mode 100644 index 0000000..079760c --- /dev/null +++ b/.github/aider/guides/sonr-service.md @@ -0,0 +1,91 @@ +# `x/svc` + +The svc module is responsible for managing the registration and authorization of services within the Sonr ecosystem. It provides a secure and verifiable mechanism for registering and authorizing services using Decentralized Identifiers (DIDs). + +## Concepts + +- **Service**: A decentralized svc on the Sonr Blockchain with properties such as ID, authority, origin, name, description, category, tags, and expiry height. +- **Profile**: Represents a DID alias with properties like ID, subject, origin, and controller. +- **Metadata**: Contains information about a svc, including name, description, category, icon, and tags. + +### Dependencies + +- [x/did](https://github.com/onsonr/sonr/tree/master/x/did) +- [x/group](https://github.com/onsonr/sonr/tree/master/x/group) +- [x/nft](https://github.com/onsonr/sonr/tree/master/x/nft) + +## State + +The module uses the following state structures: + +### Metadata + +Stores information about services: + +- Primary key: `id` (auto-increment) +- Unique index: `origin` +- Fields: id, origin, name, description, category, icon (URI), tags + +### Profile + +Stores DID alias information: + +- Primary key: `id` +- Unique index: `subject,origin` +- Fields: id, subject, origin, controller + +## Messages + +### MsgUpdateParams + +Updates the module parameters. Can only be executed by the governance account. + +### MsgRegisterService + +Registers a new svc on the blockchain. Requires a valid TXT record in DNS for the origin. + +## Params + +The module has the following parameters: + +- `categories`: List of allowed svc categories +- `types`: List of allowed svc types + +## Query + +The module provides the following query: + +### Params + +Retrieves all parameters of the module. + +## Client + +### gRPC + +The module provides a gRPC Query svc with the following RPC: + +- `Params`: Get all parameters of the module + +### CLI + +(TODO: Add CLI commands for interacting with the module) + +## Events + +(TODO: List and describe event tags used by the module) + +## Future Improvements + +- Implement svc discovery mechanisms +- Add support for svc reputation and rating systems +- Enhance svc metadata with more detailed information +- Implement svc update and deactivation functionality + +## Tests + +(TODO: Add acceptance tests for the module) + +## Appendix + +This module is part of the Sonr blockchain project and interacts with other modules such as DID and NFT modules to provide a comprehensive decentralized svc ecosystem. diff --git a/.github/aider/guides/sonr-token.md b/.github/aider/guides/sonr-token.md new file mode 100644 index 0000000..e69de29 diff --git a/.github/aider/guides/ucan-spec.md b/.github/aider/guides/ucan-spec.md new file mode 100644 index 0000000..79c5745 --- /dev/null +++ b/.github/aider/guides/ucan-spec.md @@ -0,0 +1,873 @@ +# User Controlled Authorization Network (UCAN) Specification + +# Abstract + +User-Controlled Authorization Network (UCAN) is a [trustless], secure, [local-first], user-originated, distributed authorization scheme. This document provides a high level overview of the components of the system, concepts, and motivation. Exact formats are given in [sub-specifications]. + +# Introduction + +User-Controlled Authorization Network (UCAN) is a [trustless], secure, [local-first], user-originated, distributed authorization scheme. It provides public-key verifiable, delegable, expressive, openly extensible [capabilities]. UCANs achieve public verifiability with late-bound certificate chains and principals represented by [decentralized identifiers (DIDs)][DID]. + +UCAN improves the familiarity and adoptability of schemes like [SPKI/SDSI][SPKI] for web and native application contexts. UCAN allows for the creation, delegation, and invocation of authority by any agent with a DID, including traditional systems and peer-to-peer architectures beyond traditional cloud computing. + +## Motivation + +> If we practice our principles, we could have both security and functionality. Treating security as a separate concern has not succeeded in bridging the gap between principle and practice, because it operates without knowledge of what constitutes least authority. +> +> — [Miller][Mark Miller] et al, [The Structure of Authority] + +Since at least [Multics], access control lists ([ACL]s) have been the most popular form of digital authorization, where a list of what each user is allowed to do is maintained on the resource. ACLs (and later [RBAC]) have been a successful model suited to architectures where persistent access to a single list is viable. ACLs require that rules are sufficiently well specified, such as in a centralized database with rules covering all possible permutations of scenario. This both imposes a very high maintenance burden on programmers as a systems grows in complexity, and is a key vector for [confused deputies][confused deputy problem]. + +With increasing interconnectivity between machines becoming commonplace, authorization needs to scale to meet the load demands of distributed systems while providing partition tolerance. However, it is not always practical to maintain a single central authorization source. Even when copies of the authorization list are distributed to the relevant servers, latency and partitions introduce troublesome challenges with conflicting updates, to say nothing of storage requirements. + +A large portion of personal information now also moves through connected systems. As a result, data privacy is a prominent theme when considering the design of modern applications, to the point of being legislated in parts of the world. + +Ahead-of-time coordination is often a barrier to development in many projects. Flexibility to define specialized authorization semantics for resources and the ability to integrate with external systems trustlessly are essential as the number of autonomous, specialized, and coordinated applications increases. + +Many high-value applications run in hostile environments. In recognition of this, many vendors now include public key functionality, such as [non-extractable keys in browsers][browser api crypto key], [certificate systems for external keys][fido], [platform keys][passkey], and [secure hardware enclaves] in widespread consumer devices. + +Two related models that work exceptionally well in the above context are Simple Public Key Infrastructure ([SPKI][spki rfc]) and object capabilities ([OCAP]). Since offline operation and self-verifiability are two requirements, UCAN adopts a [certificate capability model] related to [SPKI]. + +## Intuition for Auth System Differences + +The following analogies illustrate several significant trade-offs between these systems but are only accurate enough to build intuition. A good resource for a more thorough presentation of these trade-offs is [Capability Myths Demolished]. In this framework, UCAN approximates SPKI with some dynamic features. + +### Access Control Lists + +By analogy, ACLs are like a bouncer at an exclusive event. This bouncer has a list attendees allowed in and which of those are VIPs that get extra access. People trying to get in show their government-issued ID and are accepted or rejected. In addition, they may get a lanyard to identify that they have previously been allowed in. If someone is disruptive, they can simply be crossed off the list and denied further entry. + +If there are many such events at many venues, the organizers need to coordinate ahead of time, denials need to be synchronized, and attendees need to show their ID cards to many bouncers. The likelihood of the bouncer letting in the wrong person due to synchronization lag or confusion by someone sharing a name is nonzero. + +### Certificate Capabilities + +UCANs work more like [movie tickets][caps as keys] or a festival pass. No one needs to check your ID; who you are is irrelevant. For example, if you have a ticket issued by the theater to see Citizen Kane, you are admitted to Theater 3. If you cannot attend an event, you can hand this ticket to a friend who wants to see the film instead, and there is no coordination required with the theater ahead of time. However, if the theater needs to cancel tickets for some reason, they need a way of uniquely identifying them and sharing this information between them. + +### Object Capabilities + +Object capability ("ocap") systems use a combination of references, encapsulated state, and proxy forwarding. As the name implies, this is fairly close to object-oriented or actor-based systems. Object capabilities are [robust][Robust Composition], flexible, and expressive. + +To achieve these properties, object capabilities have two requirements: [fail-safe], and locality preservation. The emphasis on consistency rules out partition tolerance[^pcec]. + +## Security Considerations + +Each UCAN includes an assertions of what it is allowed to do. "Proofs" are positive evidence (elsewhere called "witnesses") of the possession of rights. They are cryptographically verifiable chains showing that the UCAN issuer either claims to directly own a resource, or that it was delegated to them by some claimed owner. In the most common case, the root owner's ID is the only globally unique identity for the resource. + +Root capability issuers function as verifiable, distributed roots of trust. The delegation chain is by definition a provenance log. Private keys themselves SHOULD NOT move from one context to another. Keeping keys unique to each physical device and unique per use case is RECOMMENDED to reduce opportunity for keys to leak, and limit blast radius in the case of compromises. "Sharing authority without sharing keys" is provided by capabilities, so there is no reason to share keys directly. + +Note that a structurally and cryptographically valid UCAN chain can be semantically invalid. The executor MUST verify the ownership of any external resources at execution time. While not possible for all use cases (e.g. replicated state machines and eventually consistent data), having the Executor be the resource itself is RECOMMENDED. + +While certificate chains go a long way toward improving security, they do not provide [confinement] on their own. The principle of least authority SHOULD be used when delegating a UCAN: minimizing the amount of time that a UCAN is valid for and reducing authority to the bare minimum required for the delegate to complete their task. This delegate should be trusted as little as is practical since they can further sub-delegate their authority to others without alerting their delegator. UCANs do not offer confinement (as that would require all processes to be online), so it is impossible to guarantee knowledge of all of the sub-delegations that exist. The ability to revoke some or all downstream UCANs exists as a last resort. + +## Inversion of Control + +[Inversion of control] is achieved due to two properties: self-certifying delegation and reference passing. There is no Authorization Server (AS) that sits between requestors and resources. In traditional terms, the owner of a UCAN resource is the resource server (RS) directly. + +This inverts the usual relationship between resources and users: the resource grants some (or all) authority over itself to agents, as opposed to an Authorization Server managing the relationship between them. This has several major advantages: + +- Fully distributed and scalable +- Self-contained request without intermediary +- Partition tolerance, [support for replicated data and machines][overcoming SSI] +- Flexible granularity +- Compositionality: no distinction between resources residing together or apart + +``` +┌─────────────┐ ┌─────────────┐ ┌─────────────┐ +│ │ │ │ │ │ +│ │ │ ┌─────────┐ │ │ │ +│ │ │ │ Bob's │ │ │ │ +│ │ │ │ Photo │ │ │ │ +│ │ │ │ Gallery │ │ │ │ +│ │ │ └─────────┘ │ │ │ +│ │ │ │ │ │ +│ Alice's │ │ Bob's │ │ Carol's │ +│ Stuff │ │ Stuff │ │ Stuff │ +│ │ │ │ │ │ +│ ┌───────┼───┼─────────────┼───┼──┐ │ +│ │ │ │ │ │ │ │ +│ │ │ │ ┌───┼───┼──┼────────┐ │ +│ │ │ │ Alice's │ │ │ │ │ │ +│ │ │ │ Music │ │ │ │Carol's │ │ +│ │ │ │ Player │ │ │ │ Game │ │ +│ │ │ │ │ │ │ │ │ │ +│ │ │ │ └───┼───┼──┼────────┘ │ +│ │ │ │ │ │ │ │ +│ └───────┼───┼─────────────┼───┼──┘ │ +│ │ │ │ │ │ +└─────────────┘ └─────────────┘ └─────────────┘ +``` + +This additionally allows UCAN to model auth for [eventually consistent and replicated state][overcoming SSI]. + +# Roles + +There are several roles that an agent MAY assume: + +| Name | Description | +| --------- | ------------------------------------------------------------------------------------------------ | +| Agent | The general class of entities and principals that interact with a UCAN | +| Audience | The Principal delegated to in the current UCAN. Listed in the `aud` field | +| Executor | The Agent that actually performs the action described in an invocation | +| Invoker | A Principal that requests an Executor perform some action that uses the Invoker's authority | +| Issuer | The Principal of the current UCAN. Listed in the `iss` field | +| Owner | A Subject that controls some external resource | +| Principal | An agent identified by DID (listed in a UCAN's `iss` or `aud` field) | +| Revoker | The Issuer listed in a proof chain that revokes a UCAN | +| Subject | The Principal who's authority is delegated or invoked | +| Validator | Any Agent that interprets a UCAN to determine that it is valid, and which capabilities it grants | + +```mermaid +flowchart TD + subgraph Agent + subgraph Principal + direction TB + + subgraph Issuer + direction TB + + subgraph Subject + direction TB + + Executor + Owner + end + + Revoker + end + + subgraph Audience + Invoker + end + end + + Validator + end +``` + +## Subject + +> At the very least every object should have a URL +> +> — [Alan Kay], [The computer revolution hasn't happened yet] + +> Every Erlang process in the universe should be addressable and introspective +> +> — [Joe Armstrong], [Code Mesh 2016] + +A [Subject] represents the Agent that a capability is for. A Subject MUST be referenced by [DID]. This behaves much like a [GUID], with the addition of public key verifiability. This unforgeability prevents malicious namespace collisions which can lead to [confused deputies][confused deputy problem]. + +### Resource + +A resource is some data or process that can be uniquely identified by a [URI]. It can be anything from a row in a database, a user account, storage quota, email address, etc. Resource MAY be as coarse or fine grained as desired. Finer-grained is RECOMMENDED where possible, as it is easier to model the principle of least authority ([PoLA]). + +A resource describes the noun of a capability. The resource pointer MUST be provided in [URI] format. Arbitrary and custom URIs MAY be used, provided that the intended recipient can decode the URI. The URI is merely a unique identifier to describe the pointer to — and within — a resource. + +Having a unique agent represent a resource (and act as its manager) is RECOMMENDED. However, to help traditional ACL-based systems transition to certificate capabilities, an agent MAY manage multiple resources, and [act as the registrant in the ACL system][wrapping existing systems]. + +Unless explicitly stated, the Resource of a UCAN MUST be the Subject. + +## Issuer & Audience + +The Issuer (`iss`) and Audience (`aud`) can be conceptualized as the sender and receiver (respectively) of a postal letter. Every UCAN MUST be signed with the private key associated with the DID in the `iss` field. + +For example: + +```js +"aud": "did:key:z6MkiTBz1ymuepAQ4HEHYSF1H8quG5GLVVQR3djdX3mDooWp", +"iss": "did:key:zDnaerDaTF5BXEavCrfRZEk316dpbLsfPDZ3WJ5hRTPFU2169", +``` + +Please see the [Cryptosuite] section for more detail on DIDs. + +# Lifecycle + +The UCAN lifecycle has four components: + +| Spec | Description | Requirement Level | +| ------------ | ------------------------------------------------------------------------ | ----------------- | +| [Delegation] | Pass, attenuate, and secure authority in a partition-tolerant way | REQUIRED | +| [Invocation] | Exercise authority that has been delegated through one or more delegates | REQUIRED | +| [Promise] | Await the result of an Invocation inside another Invocation | RECOMMENDED | +| [Revocation] | Undo a delegation, breaking a delegation chain for malicious users | RECOMMENDED | + +```mermaid +flowchart TD + prm(Promise) + inv(Invocation) + del(Delegation) + rev(Revocation) + + prm -->|awaits| inv + del -->|proves| inv + rev -.->|kind of| inv + rev -->|invalidates| del + + click del href "https://github.com/ucan-wg/delegation" "UCAN Delegation Spec" + click inv href "https://github.com/ucan-wg/invocation" "UCAN Invocation Spec" + click rev href "https://github.com/ucan-wg/revocation" "UCAN Revocation Spec" +``` + +## Time + +It is often useful to talk about a UCAN in the context of some action. For example, a UCAN delegation may be valid when it was created, but expired when invoked. + +```mermaid +sequenceDiagram + Alice -->> Bob: Delegate + Bob ->> Bob: Validate + Bob -->> Carol: Delegate + Carol ->> Carol: Validate + Carol ->> Alice: Invoke + Alice ->> Alice: Validate + Alice ->> Alice: Execute +``` + +### Validity Interval + +The period of time that a capability is valid from and until. This is the range from the latest "not before" to the earliest expiry in the UCAN delegation chain. + +### Delegation-Time + +The moment at which a delegation is asserted. This MAY be captured by an `iat` field, but is generally superfluous to capture in the token. + +### Invocation-Time + +The moment a UCAN Invocation is created. It must be within the Validity Interval. + +### Validation-Time + +Validation MAY occur at multiple points during a UCAN's lifecycle. The main two are: + +- On receipt of a delegation +- When executing an invocation + +### Execution-Time + +To avoid the overloaded word "runtime", UCAN adopts the term "execution-time" to express the moment that the executor attempts to use the authority captured in an invocation and associated delegation chain. Validation MUST occur at this time. + +## Time Bounds + +`nbf` and `exp` stand for "not before" and "expires at," respectively. These MUST be expressed as seconds since the Unix epoch in UTC, without time zone or other offset. Taken together, they represent the time bounds for a token. These timestamps MUST be represented as the number of integer seconds since the Unix epoch. Due to limitations[^js-num-size] in numerics for certain common languages, timestamps outside of the range from $-2^{53} – 1$ to $2^{53} – 1$ MUST be rejected as invalid. + +The `nbf` field is OPTIONAL. When omitted, the token MUST be treated as valid beginning from the Unix epoch. Setting the `nbf` field to a time in the future MUST delay invoking a UCAN. For example, pre-provisioning access to conference materials ahead of time but not allowing access until the day it starts is achievable with judicious use of `nbf`. + +The `exp` field is RECOMMENDED. Following the [principle of least authority][PoLA], it is RECOMMENDED to give a timestamp expiry for UCANs. If the token explicitly never expires, the `exp` field MUST be set to `null`. If the time is in the past at validation time, the token MUST be treated as expired and invalid. + +Keeping the window of validity as short as possible is RECOMMENDED. Limiting the time range can mitigate the risk of a malicious user abusing a UCAN. However, this is situationally dependent. It may be desirable to limit the frequency of forced reauthorizations for trusted devices. Due to clock drift, time bounds SHOULD NOT be considered exact. A buffer of ±60 seconds is RECOMMENDED. + +Several named points of time in the UCAN lifecycle can be found in the [high level spec][UCAN]. + +Below are a couple examples: + +```js +{ + // ... + "nbf": 1529496683, + "exp": 1575606941 +} +``` + +```js +{ + // ... + "exp": 1575606941 +} +``` + +```js +{ + // ... + "nbf": 1529496683, + "exp": null +} +``` + +## Lifecycle Example + +Here is a concrete example of all stages of the UCAN lifecycle for database write access. + +```mermaid +sequenceDiagram + participant Database + + actor DBAgent + actor Alice + actor Bob + + Note over Database, DBAgent: Set Up Agent-Owned Resource + DBAgent ->> Database: createDB() + + autonumber 1 + + Note over DBAgent, Bob: Delegation + DBAgent -->> Alice: delegate(DBAgent, write) + Alice -->> Bob: delegate(DBAgent, write) + + Note over Database, Bob: Invocation + Bob ->> DBAgent: invoke(DBAgent, [write, [key, value]], proof: [➊,➋]) + DBAgent ->> Database: write(key, value) + DBAgent ->> Bob: ACK + + Note over DBAgent, Bob: Revocation + Alice ->> DBAgent: revoke(➋, proof: [➊,➋]) + Bob ->> DBAgent: invoke(DBAgent, [write, [key, newValue]], proof: [➊,➋]) + DBAgent -X Bob: NAK(➏) [rejected] +``` + +## Capability + +A capability is the association of an ability to a subject: `subject x command x policy`. + +The Subject and Command fields are REQUIRED. Any non-normative extensions are OPTIONAL. + +For example, a capability may used to represent the ability to send email from a certain address to others at `@example.com`. + +| Field | Example | +| ------- | -------------------------------------------------------------------------------------------- | +| Subject | `did:key:z6MkhaXgBZDvotDkL5257faiztiGiC2QtKLGpbnnEGta2doK` | +| Command | `/msg/send` | +| Policy | `["or", ["==", ".from", "mailto:me@example.com"], ["match", ".cc", "mailto:*@example.com"]]` | + +For a more complete treatment, please see the [UCAN Delegation][delegation] spec. + +## Authority + +> Whether to enable cooperation or to limit vulnerability, we care about _authority_ rather than _permissions._ Permissions determine what actions an individual program may perform on objects it can directly access. Authority describes the effects that a program may cause on objects it can access, either directly by permission, or indirectly by permitted interactions with other programs. +> +> —[Mark Miller], [Robust Composition] + +The set of capabilities delegated by a UCAN is called its "authority." To frame it another way, it's the set of effects that a principal can cause, and acts as a declarative description of delegated abilities. + +Merging capability authorities MUST follow set semantics, where the result includes all capabilities from the input authorities. Since broader capabilities automatically include narrower ones, this process is always additive. Capability authorities can be combined in any order, with the result always being at least as broad as each of the original authorities. + +```plaintext + ┌───────────────────────┐ ┐ + │ │ │ + │ │ │ + │ │ │ + │ │ │ + │ Subject B │ │ +┌──────────────────┼ ─ ─ x │ │ +│ │ Ability Z │ ├── BxZ +│ │ │ │ Capability +│ │ │ │ +│ │ │ │ +│ Subject A │ │ │ +│ x │ │ │ +│ Ability Y ─ ─┼──────────────────┘ ┘ +│ │ +│ │ +│ │ +│ │ +│ │ +└───────────────────────┘ + +└─────────────────────┬────────────────────┘ + │ + AxY U BxZ + Capability +``` + +The capability authority is the total rights of the authorization space down to the relevant volume of authorizations. Individual capabilities MAY overlap; the authority is the union. Every unique delegated capability MUST have equal or narrower capabilities from their delegator. Inside this content space, you can draw a boundary around some resource(s) (their type, identifiers, and paths or children) and their capabilities. + +## Command + +Commands are concrete messages ("verbs") that MUST be unambiguously interpretable by the Subject of a UCAN. Commands are REQUIRED in invocations. Some examples include `/msg/send`, `/crud/read`, and `/ucan/revoke`. + +Much like other message-passing systems, the specific resource MUST define the behavior for a particular message. For instance, `/crud/update` MAY be used to destructively update a database row, or append to a append-only log. Specific messages MAY be created at will; the only restriction is that the Executor understand how to interpret that message in the context of a specific resource. + +While arbitrary semantics MAY be described, they MUST apply to the target resource. For instance, it does not make sense to apply `/msg/send` to a typical file system. + +### Segment Structure + +Commands MUST be lowercase, and begin with a slash (`/`). Segments MUST be separated by a slash. A trailing slash MUST NOT be present. All of the following are syntactically valid Commands: + +- `/` +- `/crud` +- `/crud/create` +- `/stack/pop` +- `/crypto/sign` +- `/foo/bar/baz/qux/quux` +- `/ほげ/ふが` + +Segment structure is important since shorter Commands prove longer paths. For example, `/` can be used as a proof of _any_ other Command. For example, `/crypto` MAY be used to prove `/crypto/sign` but MUST NOT prove `/stack/pop` or `/cryptocurrency`. + +### `/` AKA "Top" + +_"Top" (`/`) is the most powerful ability, and as such it SHOULD be handled with care and used sparingly._ + +The "top" (or "any", or "wildcard") ability MUST be denoted `/`. This can be thought of as something akin to a super user permission in RBAC. + +The wildcard ability grants access to all other capabilities for the specified resource, across all possible namespaces. The wildcard ability is useful when "linking" agents by delegating all access to another device controlled by the same user, and that should behave as the same agent. It is extremely powerful, and should be used with care. Among other things, it permits the delegate to update a Subject's mutable DID document (change their private keys), revoke UCAN delegations, and use any resources delegated to the Subject by others. + +```mermaid +%%{ init: { 'flowchart': { 'curve': 'linear' } } }%% + +flowchart BT + / + + /msg --> / + subgraph msgGraph [ ] + /msg/send --> /msg + /msg/receive --> /msg + end + + /crud --> / + subgraph crudGraph [ ] + /crud/read --> /crud + /crud/mutate --> /crud + + subgraph mutationGraph [ ] + /crud/mutate/create --> /crud/mutate + /crud/mutate/update --> /crud/mutate + /crud/mutate/destroy --> /crud/mutate + end + end + + ... --> / +``` + +### Reserved Commands + +#### `/ucan` Namespace + +The `/ucan` Command namespace MUST be reserved. This MUST include any ability string matching the regex `^ucan\/.*`. This is important for keeping a space for community-blessed Commands in the future, such as standard library Commands, such as [Revocation]. + +## Attenuation + +Attenuation is the process of constraining the capabilities in a delegation chain. Each direct delegation MUST either directly restate or attenuate (diminish) its capabilities. + +# Token Resolution + +Token resolution is transport specific. The exact format is left to the relevant UCAN transport specification. At minimum, such a specification MUST define at least the following: + +1. Request protocol +2. Response protocol +3. Collections format + +Note that if an instance cannot dereference a CID at runtime, the UCAN MUST fail validation. This is consistent with the [constructive semantics] of UCAN. + +# Nonce + +The REQUIRED nonce parameter `nonce` MAY be any value. A randomly generated string is RECOMMENDED to provide a unique UCAN, though it MAY also be a monotonically increasing count of the number of links in the hash chain. This field helps prevent replay attacks and ensures a unique CID per delegation. The `iss`, `aud`, and `exp` fields together will often ensure that UCANs are unique, but adding the nonce ensures uniqueness. + +The recommended size of the nonce differs by key type. In many cases, a random 12-byte nonce is sufficient. If uncertain, check the nonce in your DID's crypto suite. + +This field SHOULD NOT be used to sign arbitrary data, such as signature challenges. See the [`meta`][Metadata] field for more. + +Here is a simple example. + +```js +{ + // ... + "nonce": {"/": {"bytes": "bGlnaHQgd29yay4"}} +} +``` + +# Metadata + +The OPTIONAL `meta` field contains a map of arbitrary metadata, facts, and proofs of knowledge. The enclosed data MUST be self-evident and externally verifiable. It MAY include information such as hash preimages, server challenges, a Merkle proof, dictionary data, etc. + +The data contained in this map MUST NOT be semantically meaningful to delegation chains. + +Below is an example: + +```js +{ + // ... + "meta": { + "challenges": { + "example.com": "abcdef", + "another.example.net": "12345" + }, + "sha3_256": { + "B94D27B9934D3E08A52E52D7DA7DABFAC484EFE37A5380EE9088F7ACE2EFCDE9": "hello world" + } + } +} +``` + +# Canonicalization + +## Cryptosuite + +Across all UCAN specifications, the following cryptosuite MUST be supported: + +| Role | REQUIRED Algorithms | Notes | +| --------- | --------------------------------- | ------------------------------------ | +| Hash | [SHA-256] | | +| Signature | [Ed25519], [P-256], [`secp256k1`] | Preference of Ed25519 is RECOMMENDED | +| [DID] | [`did:key`] | | + +## Encoding + +All UCANs MUST be canonically encoded with [DAG-CBOR] for signing. A UCAN MAY be presented or stored in other [IPLD] formats (such as [DAG-JSON]), but converted to DAG-CBOR for signature validation. + +## Content Identifiers + +A UCAN token MUST be configured as follows: + +| Parameter | REQUIRED Configuration | +| ------------ | ---------------------- | +| Version | [CIDv1] | +| [Multibase] | [`base58btc`] | +| [Multihash] | [SHA-256] | +| [Multicodec] | [DAG-CBOR] | + +> [!NOTE] +> All CIDs encoded as above start with the characters `zdpu`. + +The resolution of these addresses is left to the implementation and end-user, and MAY (non-exclusively) include the following: local store, a distributed hash table (DHT), gossip network, or RESTful service. + +## Envelope + +All UCAN formats MUST use the following envelope format: + +| Field | Type | Description | +| --------------------------------- | -------------- | -------------------------------------------------------------- | +| `.0` | `Bytes` | A signature by the Payload's `iss` over the `SigPayload` field | +| `.1` | `SigPayload` | The content that was signed | +| `.1.h` | `VarsigHeader` | The [Varsig] v1 header | +| `.1.ucan/@` | `TokenPayload` | The UCAN token payload | + +```mermaid +flowchart TD + subgraph Ucan ["UCAN Envelope"] + SignatureBytes["Signature (raw bytes)"] + + subgraph SigPayload ["Signature Payload"] + VarsigHeader["Varsig Header"] + + subgraph UcanPayload ["Token Payload"] + fields["..."] + end + end + end +``` + +For example: + +```js +[ + { + "/": { + bytes: + "7aEDQLYvb3lygk9yvAbk0OZD0q+iF9c3+wpZC4YlFThkiNShcVriobPFr/wl3akjM18VvIv/Zw2LtA4uUmB5m8PWEAU", + }, + }, + { + h: { "/": { bytes: "NBIFEgEAcQ" } }, + "ucan/example@1.0.0-rc.1": { + hello: "world", + }, + }, +]; +``` + +### Payload + +A UCAN's Payload MUST contain at least the following fields: + +| Field | Type | Required | Description | +| ------- | ----------------------------------------- | -------- | ----------------------------------------------------------- | +| `iss` | `DID` | Yes | Issuer DID (sender) | +| `aud` | `DID` | Yes | Audience DID (receiver) | +| `sub` | `DID` | Yes | Principal that the chain is about (the [Subject]) | +| `cmd` | `String` | Yes | The [Command] to eventually invoke | +| `args` | `{String : Any}` | Yes | Any [Arguments] that MUST be present in the Invocation | +| `nonce` | `Bytes` | Yes | Nonce | +| `meta` | `{String : Any}` | No | [Meta] (asserted, signed data) — is not delegated authority | +| `nbf` | `Integer` (53-bits[^js-num-size]) | No | "Not before" UTC Unix Timestamp in seconds (valid from) | +| `exp` | `Integer \| Null` (53-bits[^js-num-size]) | Yes | Expiration UTC Unix Timestamp in seconds (valid until) | + +# Implementation Recommendations + +## Delegation Store + +A validator MAY keep a local store of UCANs that it has received. UCANs are immutable but also time-bound so that this store MAY evict expired or revoked UCANs. + +This store SHOULD be indexed by CID (content addressing). Multiple indices built on top of this store MAY be used to improve capability search or selection performance. + +## Memoized Validation + +Aside from revocation, capability validation is idempotent. Marking a CID (or capability index inside that CID) as valid acts as memoization, obviating the need to check the entire structure on every validation. This extends to distinct UCANs that share a proof: if the proof was previously reviewed and is not revoked, it is RECOMMENDED to consider it valid immediately. + +Revocation is irreversible. Suppose the validator learns of revocation by UCAN CID. In that case, the UCAN and all of its derivatives in such a cache MUST be marked as invalid, and all validations immediately fail without needing to walk the entire structure. + +## Replay Attack Prevention + +Replay attack prevention is REQUIRED. Every UCAN token MUST hash to a unique [CIDv1]. Some simple strategies for implementing uniqueness tracking include maintaining a set of previously seen CIDs, or requiring that nonces be monotonically increasing per principal. This MAY be the same structure as a validated UCAN memoization table (if one is implemented). + +Maintaining a secondary token expiry index is RECOMMENDED. This enables garbage collection and more efficient search. In cases of very large stores, normal cache performance techniques MAY be used, such as Bloom filters, multi-level caches, and so on. + +## Beyond Single System Image + +> As we continue to increase the number of globally connected devices, we must embrace a design that considers every single member in the system as the primary site for the data that it is generates. It is completely impractical that we can look at a single, or a small number, of globally distributed data centers as the primary site for all global information that we desire to perform computations with. +> +> —[Meiklejohn], [A Certain Tendency Of The Database Community] + +Unlike many authorization systems where a service controls access to resources in their care, location-independent, offline, and leaderless resources require control to live with the user. Therefore, the same data MAY be used across many applications, data stores, and users. Since they don't have a single location, applying UCAN to [RSM]s and [CRDT]s MAY be modelled by lifting the requirement that the Executor be the Subject. + +Ultimately this comes down to a question of push vs pull. In push, the subject MUST be the specific site being pushed to ("I command you to apply the following updates to your state"). + +Pull is the broad class of situations where an Invoker doesn't require that a particular replica apply its state. Applying a change to a local CRDT replica and maintaining a UCAN invocation log is a valid update to "the CRDT": a version of the CRDT Subject exists locally even if the Subject's private key is not present. Gossiping these changes among agents allows each to apply changes that it becomes aware of. Thanks to the invocation log (or equivalent integrated directly into the CRDT), provenance of authority is made transparent. + +```mermaid +sequenceDiagram + participant CRDT as Initial Grow-Only Set (CRDT) + + actor Alice + actor Bob + actor Carol + + autonumber + + Note over CRDT, Bob: Setup + CRDT -->> Alice: delegate(CRDT_ID, merge) + CRDT -->> Bob: delegate(CRDT_ID, merge) + + Note over Bob, Carol: Bob Invites Carol + Bob -->> Carol: delegate(CRDT_ID, merge) + + Note over Alice, Carol: Direct P2P Gossip + Carol ->> Bob: invoke(CRDT_ID, merge, {"Carrot"}, proof: [➋,❸]) + Alice ->> Carol: invoke(CRDT_ID, merge, {"Apple"}}, proof: [➊]) + Bob ->> Alice: invoke(CRDT_ID, merge, {"Banana", "Carrot"}, proof: [➋]) +``` + +## Wrapping Existing Systems + +In the RECOMMENDED scenario, the agent controlling a resource has a unique reference to it. This is always possible in a system that has adopted capabilities end-to-end. + +Interacting with existing systems MAY require relying on ambient authority contained in an ACL, non-unique reference, or other authorization logic. These cases are still compatible with UCAN, but the security guarantees are weaker since 1. the surface area is larger, and 2. part of the auth system lives outside UCAN. + +```mermaid +sequenceDiagram + participant Database + participant ACL as External Auth System + + actor DBAgent + actor Alice + actor Bob + + Note over ACL, DBAgent: Setup + DBAgent ->> ACL: signup(DBAgent) + ACL ->> ACL: register(DBAgent) + + autonumber 1 + + Note over DBAgent, Bob: Delegation + DBAgent -->> Alice: delegate(DBAgent, write) + Alice -->> Bob: delegate(DBAgent, write) + + Note over Database, Bob: Invocation + Bob ->>+ DBAgent: invoke(DBAgent, [write, key, value], proof: [➊,➋]) + + critical External System + DBAgent ->> ACL: getToken(write, key, AuthGrant) + ACL ->> DBAgent: AccessToken + + DBAgent ->> Database: request(write, value, AccessToken) + Database ->> DBAgent: ACK + end + + DBAgent ->>- Bob: ACK +``` + +# FAQ + +## What prevents an unauthorized party from using an intercepted UCAN? + +UCANs always contain information about the sender and receiver. A UCAN is signed by the sender (the `iss` field DID) and can only be created by an agent in possession of the relevant private key. The recipient (the `aud` field DID) is required to check that the field matches their DID. These two checks together secure the certificate against use by an unauthorized party. [UCAN Invocations][invocation] prevent use by an unauthorized party by signing over a request to use the capability granted in a delegation chain. + +## What prevents replay attacks on the invocation use case? + +All UCAN Invocations MUST have a unique CID. The executing agent MUST check this validation uniqueness against a local store of unexpired UCAN hashes. + +This is not a concern when simply delegating since receiving a delegation is idempotent. + +## Is UCAN secure against person-in-the-middle attacks? + +_UCAN does not have any special protection against person-in-the-middle (PITM) attacks._ + +If a PITM attack was successfully performed on a UCAN delegation, the proof chain would contain the attacker's DID(s). It is possible to detect this scenario and revoke the relevant UCAN but this does require special inspection of the topmost `iss` field to check if it is the expected DID. Therefore, it is strongly RECOMMENDED to only delegate UCANs to agents that are both trusted and authenticated and over secure channels. + +## Can my implementation support more cryptographic algorithms? + +It is possible to use other algorithms, but doing so limits interoperability with the broader UCAN ecosystem. This is thus considered "off spec" (i.e. non-interoperable). If you choose to extend UCAN with additional algorithms, you MUST include this metadata in the (self-describing) [Varsig] header. + +# Related Work and Prior Art + +[SPKI/SDSI] is closely related to UCAN. A different encoding format is used, and some details vary (such as a delegation-locking bit), but the core idea and general usage pattern are very close. UCAN can be seen as making these ideas more palatable to a modern audience and adding a few features such as content IDs that were less widespread at the time SPKI/SDSI were written. + +[ZCAP-LD] is closely related to UCAN. The primary differences are in formatting, addressing by URL instead of CID, the mechanism of separating invocation from authorization, and single versus multiple proofs. + +[CACAO] is a translation of many of these ideas to a cross-blockchain delegated bearer token model. It contains the same basic concepts as UCAN delegation, but is aimed at small messages and identities that are rooted in mutable documents rooted on a blockchain and lacks the ability to subdelegate capabilities. + +[Local-First Auth] is a non-certificate-based approach, instead relying on a CRDT to build up a list of group members, devices, and roles. It has a friendly invitation mechanism based on a [Seitan token exchange]. It is also straightforward to see which users have access to what, avoiding the confinement problem seen in many decentralized auth systems. + +[Macaroon] is a MAC-based capability and cookie system aimed at distributing authority across services in a trusted network (typically in the context of a Cloud). By not relying on asymmetric signatures, Macaroons achieve excellent space savings and performance, given that the MAC can be checked against the relevant services during discharge. The authority is rooted in an originating server rather than with an end-user. + +[Biscuit] uses Datalog to describe capabilities. It has a specialized format but is otherwise in line with UCAN. + +[Verifiable credentials] are a solution for data about people or organizations. However, they are aimed at a related-but-distinct problem: asserting attributes about the holder of a DID, including things like work history, age, and membership. + +# Acknowledgments + +Thank you to [Brendan O'Brien] for real-world feedback, technical collaboration, and implementing the first Golang UCAN library. + +Thank you [Blaine Cook] for the real-world feedback, ideas on future features, and lessons from other auth standards. + +Many thanks to [Hugo Dias], [Mikael Rogers], and the entire DAG House team for the real world feedback, and finding inventive new use cases. + +Thank to [Hannah Howard] and [Alan Shaw] at [Storacha] for their team's feedback from real world use cases. + +Many thanks to [Brian Ginsburg] and [Steven Vandevelde] for their many copy edits, feedback from real world usage, maintenance of the TypeScript implementation, and tools such as [ucan.xyz]. + +Many thanks to [Christopher Joel] for his real-world feedback, raising many pragmatic considerations, and the Rust implementation and related crates. + +Many thanks to [Christine Lemmer-Webber] for her handwritten(!) feedback on the design of UCAN, spearheading the [OCapN] initiative, and her related work on [ZCAP-LD]. + +Many thanks to [Alan Karp] for sharing his vast experience with capability-based authorization, patterns, and many right words for us to search for. + +Thanks to [Benjamin Goering] for the many community threads and connections to [W3C] standards. + +Thanks to [Juan Caballero] for the numerous questions, clarifications, and general advice on putting together a comprehensible spec. + +Thank you [Dan Finlay] for being sufficiently passionate about [OCAP] that we realized that capability systems had a real chance of adoption in an ACL-dominated world. + +Thanks to [Peter van Hardenberg][PvH] and [Martin Kleppmann] of [Ink & Switch] for conversations exploring options for access control on CRDTs and [local-first] applications. + +Thanks to the entire [SPKI WG][SPKI/SDSI] for their closely related pioneering work. + +We want to especially recognize [Mark Miller] for his numerous contributions to the field of distributed auth, programming languages, and networked security writ large. + + + +[^js-num-size]: JavaScript has a single numeric type ([`Number`][JS Number]) for both integers and floats. This representation is defined as a [IEEE-754] double-precision floating point number, which has a 53-bit significand. + +[^pcec]: To be precise, this is a [PC/EC][PACELC] system, which is a critical trade-off for many systems. UCAN can be used to model both PC/EC and PA/EL, but is most typically PC/EL. + + + +[Command]: #command +[Cryptosuite]: #cryptosuite +[overcoming SSI]: #beyond-single-system-image +[sub-specifications]: #sub-specifications +[wrapping existing systems]: #wrapping-existing-systems + + + +[IEEE-754]: https://ieeexplore.ieee.org/document/8766229 +[A Certain Tendency Of The Database Community]: https://arxiv.org/pdf/1510.08473.pdf +[ACL]: https://en.wikipedia.org/wiki/Access-control_list +[Alan Karp]: https://github.com/alanhkarp +[Alan Kay]: https://en.wikipedia.org/wiki/Alan_Kay +[Alan Shaw]: https://github.com/alanshaw +[BCP 14]: https://www.rfc-editor.org/info/bcp14 +[BLAKE3]: https://github.com/BLAKE3-team/BLAKE3 +[Benjamin Goering]: https://github.com/gobengo +[Biscuit]: https://github.com/biscuit-auth/biscuit/ +[Blaine Cook]: https://github.com/blaine +[Bluesky]: https://blueskyweb.xyz/ +[Brendan O'Brien]: https://github.com/b5 +[Brian Ginsburg]: https://github.com/bgins +[Brooklyn Zelenka]: https://github.com/expede +[CACAO]: https://blog.ceramic.network/capability-based-data-security-on-ceramic/ +[CIDv1]: https://docs.ipfs.io/concepts/content-addressing/#identifier-formats +[CIDv1]: https://github.com/multiformats/cid +[CRDT]: https://en.wikipedia.org/wiki/Conflict-free_replicated_data_type +[Capability Myths Demolished]: https://srl.cs.jhu.edu/pubs/SRL2003-02.pdf +[Christine Lemmer-Webber]: https://github.com/cwebber +[Christopher Joel]: https://github.com/cdata +[Code Mesh 2016]: https://www.codemesh.io/codemesh2016 +[DAG-CBOR]: https://ipld.io/specs/codecs/dag-cbor/spec/ +[DAG-JSON]: https://ipld.io/specs/codecs/dag-json/spec/ +[DID fragment]: https://www.w3.org/TR/did-core/#fragment +[DID path]: https://www.w3.org/TR/did-core/#path +[DID subject]: https://www.w3.org/TR/did-core/#dfn-did-subjects +[DID]: https://www.w3.org/TR/did-core/ +[Dan Finlay]: https://github.com/danfinlay +[Daniel Holmgren]: https://github.com/dholms +[ECDSA security]: https://en.wikipedia.org/wiki/Elliptic_Curve_Digital_Signature_Algorithm#Security +[Ed25519]: https://en.wikipedia.org/wiki/EdDSA#Ed25519 +[EdDSA]: https://datatracker.ietf.org/doc/html/rfc8032#section-5.1 +[Email about SPKI]: https://web.archive.org/web/20140724054706/http://wiki.erights.org/wiki/Capability-based_Active_Invocation_Certificates +[FIDO]: https://fidoalliance.org/what-is-fido/ +[Fission]: https://fission.codes +[GUID]: https://en.wikipedia.org/wiki/Universally_unique_identifier +[Hannah Howard]: https://github.com/hannahhoward +[Hugo Dias]: https://github.com/hugomrdias +[IPLD]: https://ipld.io/ +[Ink & Switch]: https://www.inkandswitch.com/ +[Inversion of control]: https://en.wikipedia.org/wiki/Inversion_of_control +[Irakli Gozalishvili]: https://github.com/Gozala +[JWT]: https://www.rfc-editor.org/rfc/rfc7519 +[Joe Armstrong]: https://en.wikipedia.org/wiki/Joe_Armstrong_(programmer) +[Juan Caballero]: https://github.com/bumblefudge +[Local-First Auth]: https://github.com/local-first-web/auth +[Macaroon]: https://storage.googleapis.com/pub-tools-public-publication-data/pdf/41892.pdf +[Mark Miller]: https://github.com/erights +[Martin Kleppmann]: https://martin.kleppmann.com/ +[Meiklejohn]: https://christophermeiklejohn.com/ +[Mikael Rogers]: https://github.com/mikeal/ +[Multibase]: https://github.com/multiformats/multibase +[Multicodec]: https://github.com/multiformats/multicodec +[Multics]: https://en.wikipedia.org/wiki/Multics +[Multihash]: https://www.multiformats.io/multihash/ +[OCAP]: http://erights.org/elib/capability/index.html +[OCapN]: https://github.com/ocapn/ocapn +[P-256]: https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.186-4.pdf#page=111 +[PACELC]: https://en.wikipedia.org/wiki/PACELC_theorem +[Philipp Krüger]: https://github.com/matheus23 +[PoLA]: https://en.wikipedia.org/wiki/Principle_of_least_privilege +[Protocol Labs]: https://protocol.ai/ +[PvH]: https://www.pvh.ca +[RBAC]: https://en.wikipedia.org/wiki/Role-based_access_control +[RFC 2119]: https://datatracker.ietf.org/doc/html/rfc2119 +[RFC 3339]: https://www.rfc-editor.org/rfc/rfc3339 +[RFC 8037]: https://datatracker.ietf.org/doc/html/rfc8037 +[RSM]: https://en.wikipedia.org/wiki/State_machine_replication +[Robust Composition]: http://www.erights.org/talks/thesis/markm-thesis.pdf +[SHA-256]: https://en.wikipedia.org/wiki/SHA-2 +[SPKI/SDSI]: https://datatracker.ietf.org/wg/spki/about/ +[SPKI]: https://theworld.com/~cme/html/spki.html +[Seitan token exchange]: https://book.keybase.io/docs/teams/seitan +[Steven Vandevelde]: https://github.com/icidasset +[Storacha]: https://storacha.network/ +[The Structure of Authority]: http://erights.org/talks/no-sep/secnotsep.pdf +[The computer revolution hasn't happened yet]: https://www.youtube.com/watch?v=oKg1hTOQXoY +[UCAN Promise]: https://github.com/ucan-wg/promise +[URI]: https://www.rfc-editor.org/rfc/rfc3986 +[Varsig]: https://github.com/ChainAgnostic/varsig +[Verifiable credentials]: https://www.w3.org/2017/vc/WG/ +[W3C]: https://www.w3.org/ +[WebCrypto API]: https://developer.mozilla.org/en-US/docs/Web/API/Web_Crypto_API +[Witchcraft Software]: https://github.com/expede +[ZCAP-LD]: https://w3c-ccg.github.io/zcap-spec/ +[`base58btc`]: https://github.com/multiformats/multibase/blob/master/multibase.csv#L21 +[`did:key`]: https://w3c-ccg.github.io/did-method-key/ +[`secp256k1`]: https://en.bitcoin.it/wiki/Secp256k1 +[browser api crypto key]: https://developer.mozilla.org/en-US/docs/Web/API/CryptoKey +[capabilities]: https://en.wikipedia.org/wiki/Object-capability_model +[caps as keys]: http://www.erights.org/elib/capability/duals/myths.html#caps-as-keys +[certificate capability model]: https://web.archive.org/web/20140724054706/http://wiki.erights.org/wiki/Capability-based_Active_Invocation_Certificates +[confinement]: http://www.erights.org/elib/capability/dist-confine.html +[confused deputy problem]: https://en.wikipedia.org/wiki/Confused_deputy_problem +[constructive semantics]: https://en.wikipedia.org/wiki/Intuitionistic_logic +[content addressable storage]: https://en.wikipedia.org/wiki/Content-addressable_storage +[content addressing]: https://en.wikipedia.org/wiki/Content-addressable_storage +[dag-json multicodec]: https://github.com/multiformats/multicodec/blob/master/table.csv#L104 +[delegation]: https://github.com/ucan-wg/delegation +[fail-safe]: https://en.wikipedia.org/wiki/Fail-safe +[invocation]: https://github.com/ucan-wg/invocation +[local-first]: https://www.inkandswitch.com/local-first/ +[number zero]: https://n0.computer/ +[passkey]: https://www.passkeys.com/ +[promise]: https://github.com/ucan-wg/promise +[raw data multicodec]: https://github.com/multiformats/multicodec/blob/a03169371c0a4aec0083febc996c38c3846a0914/table.csv?plain=1#L41 +[revocation]: https://github.com/ucan-wg/revocation +[secure hardware enclave]: https://support.apple.com/en-ca/guide/security/sec59b0b31ff +[spki rfc]: https://www.rfc-editor.org/rfc/rfc2693.html +[time definition]: https://en.wikipedia.org/wiki/Temporal_database +[trustless]: https://blueskyweb.xyz/blog/3-6-2022-a-self-authenticating-social-protocol +[ucan.xyz]: https://ucan.xyz diff --git a/.github/aider/prompts/data-modeler-cosmos.md b/.github/aider/prompts/data-modeler-cosmos.md new file mode 100644 index 0000000..65477ef --- /dev/null +++ b/.github/aider/prompts/data-modeler-cosmos.md @@ -0,0 +1,105 @@ +You are an expert in Cosmos SDK data modeling and state management, specializing in building efficient and scalable data models using the Cosmos SDK ORM system with Protocol Buffers. + +Key Principles: + +- Design type-safe state management systems +- Create efficient protobuf-based data models +- Implement proper table structures and indexes +- Follow Cosmos SDK state management best practices +- Design for light client compatibility +- Implement proper genesis import/export +- Follow protobuf naming conventions + +Data Modeling Best Practices: + +- Define clear table structures in .proto files +- Use appropriate primary key strategies +- Implement proper secondary indexes +- Follow database normalization principles (1NF+) +- Avoid repeated fields in tables +- Design for future extensibility +- Consider state layout impact on clients + +Schema Design Patterns: + +- Use unique table IDs within .proto files +- Implement proper field numbering +- Design efficient multipart keys +- Use appropriate field types +- Consider index performance implications +- Implement proper singleton patterns +- Design for automatic query services + +State Management: + +- Follow Cosmos SDK store patterns +- Implement proper prefix handling +- Design efficient range queries +- Use appropriate encoding strategies +- Handle state migrations properly +- Implement proper genesis handling +- Consider light client proof requirements + +Error Handling and Validation: + +- Implement proper input validation +- Use appropriate error types +- Handle state errors appropriately +- Implement proper debugging +- Use context appropriately +- Implement proper logging +- Handle concurrent access + +Performance Optimization: + +- Design efficient key encodings +- Optimize storage space usage +- Implement efficient queries +- Use appropriate index strategies +- Consider state growth implications +- Monitor performance metrics +- Design for scalability + +Dependencies: + +- cosmos/orm/v1/orm.proto +- [google.golang.org/protobuf](http://google.golang.org/protobuf) +- cosmos-sdk/store +- cosmos-sdk/types +- tendermint/types +- proper logging framework + +Key Conventions: + +1. Use consistent protobuf naming +2. Implement proper documentation +3. Follow schema versioning practices +4. Use proper table ID management +5. Implement proper testing strategies + +Example Table Structure: + +```protobuf +message Balance { + option (cosmos.orm.v1.table) = { + id: 1 + primary_key: { fields: "account,denom" } + index: { id: 1, fields: "denom" } + }; + + bytes account = 1; + string denom = 2; + uint64 amount = 3; +} + +message Params { + option (cosmos.orm.v1.singleton) = { + id: 2 + }; + + google.protobuf.Duration voting_period = 1; + uint64 min_threshold = 2; +} +``` + +Refer to the official Cosmos SDK documentation and ORM specifications for best practices and up-to-date APIs. \ No newline at end of file diff --git a/.github/aider/prompts/data-modeler.md b/.github/aider/prompts/data-modeler.md new file mode 100644 index 0000000..0d2d542 --- /dev/null +++ b/.github/aider/prompts/data-modeler.md @@ -0,0 +1,88 @@ + You are an expert in Go data modeling and PostgreSQL database design, specializing in building efficient and scalable data models using modern ORMs like GORM and SQLBoiler. + + Key Principles: + - Write idiomatic Go code following standard Go conventions + - Design clean and maintainable database schemas + - Implement proper relationships and constraints + - Use appropriate indexes for query optimization + - Follow database normalization principles + - Implement proper error handling and validation + - Use meaningful struct tags for ORM mapping + + Data Modeling Best Practices: + - Use appropriate Go types for database columns + - Implement proper foreign key relationships + - Design for data integrity and consistency + - Consider soft deletes where appropriate + - Use composite indexes strategically + - Implement proper timestamps for auditing + - Handle NULL values appropriately with pointers + + ORM Patterns: + - Use GORM hooks for complex operations + - Implement proper model validation + - Use transactions for atomic operations + - Implement proper eager loading + - Use batch operations for better performance + - Handle migrations systematically + - Implement proper model scopes + + Database Design: + - Follow PostgreSQL best practices + - Use appropriate column types + - Implement proper constraints + - Design efficient indexes + - Use JSONB for flexible data when needed + - Implement proper partitioning strategies + - Consider materialized views for complex queries + + Error Handling and Validation: + - Implement proper input validation + - Use custom error types + - Handle database errors appropriately + - Implement retry mechanisms + - Use context for timeouts + - Implement proper logging + - Handle concurrent access + + Performance Optimization: + - Use appropriate batch sizes + - Implement connection pooling + - Use prepared statements + - Optimize query patterns + - Use appropriate caching strategies + - Monitor query performance + - Use explain analyze for optimization + + Dependencies: + - GORM or SQLBoiler + - pq (PostgreSQL driver) + - validator + - migrate + - sqlx (for raw SQL when needed) + - zap or logrus for logging + + Key Conventions: + 1. Use consistent naming conventions + 2. Implement proper documentation + 3. Follow database migration best practices + 4. Use version control for schema changes + 5. Implement proper testing strategies + + Example Model Structure: + ```go + type User struct { + ID uint `gorm:"primarykey"` + CreatedAt time.Time + UpdatedAt time.Time + DeletedAt gorm.DeletedAt `gorm:"index"` + + Name string `gorm:"type:varchar(100);not null"` + Email string `gorm:"type:varchar(100);uniqueIndex;not null"` + Profile Profile + Orders []Order + } + ``` + + Refer to the official documentation of GORM, PostgreSQL, and Go for best practices and up-to-date APIs. + diff --git a/.github/aider/prompts/sonr-tech-lead.md b/.github/aider/prompts/sonr-tech-lead.md new file mode 100644 index 0000000..746b271 --- /dev/null +++ b/.github/aider/prompts/sonr-tech-lead.md @@ -0,0 +1,132 @@ +You are a technical lead specializing in decentralized identity systems and security architecture, with expertise in W3C standards, Cosmos SDK, and blockchain security patterns. + +Core Responsibilities: +- Ensure compliance with W3C DID and VC specifications +- Implement secure cryptographic practices +- Design robust authentication flows +- Maintain data privacy and protection +- Guide secure state management +- Enforce access control patterns +- Oversee security testing + +Security Standards: +- W3C DID Core 1.0 +- W3C Verifiable Credentials +- W3C WebAuthn Level 2 +- OAuth 2.0 and OpenID Connect +- JSON Web Signatures (JWS) +- JSON Web Encryption (JWE) +- Decentralized Key Management (DKMS) + +Architecture Patterns: +- Secure DID Resolution +- Verifiable Credential Issuance +- DWN Access Control +- Service Authentication +- State Validation +- Key Management +- Privacy-Preserving Protocols + +Implementation Guidelines: +- Use standardized cryptographic libraries +- Implement proper key derivation +- Follow secure encoding practices +- Validate all inputs thoroughly +- Handle errors securely +- Log security events properly +- Implement rate limiting + +State Management Security: +- Validate state transitions +- Implement proper access control +- Use secure storage patterns +- Handle sensitive data properly +- Implement proper backup strategies +- Maintain state integrity +- Monitor state changes + +Authentication & Authorization: +- Implement proper DID authentication +- Use secure credential validation +- Follow OAuth 2.0 best practices +- Implement proper session management +- Use secure token handling +- Implement proper key rotation +- Monitor authentication attempts + +Data Protection: +- Encrypt sensitive data +- Implement proper key management +- Use secure storage solutions +- Follow data minimization principles +- Implement proper backup strategies +- Handle data deletion securely +- Monitor data access + +Security Testing: +- Implement security unit tests +- Perform integration testing +- Conduct penetration testing +- Monitor security metrics +- Review security logs +- Conduct threat modeling +- Maintain security documentation + +Example Security Patterns: + +```go +// Secure DID Resolution +func ResolveDID(did string) (*DIDDocument, error) { + // Validate DID format + if !ValidateDIDFormat(did) { + return nil, ErrInvalidDID + } + + // Resolve with retry and timeout + ctx, cancel := context.WithTimeout(context.Background(), resolveTimeout) + defer cancel() + + doc, err := resolver.ResolveWithContext(ctx, did) + if err != nil { + return nil, fmt.Errorf("resolution failed: %w", err) + } + + // Validate document structure + if err := ValidateDIDDocument(doc); err != nil { + return nil, fmt.Errorf("invalid document: %w", err) + } + + return doc, nil +} + +// Secure Credential Verification +func VerifyCredential(vc *VerifiableCredential) error { + // Check expiration + if vc.IsExpired() { + return ErrCredentialExpired + } + + // Verify proof + if err := vc.VerifyProof(trustRegistry); err != nil { + return fmt.Errorf("invalid proof: %w", err) + } + + // Verify status + if err := vc.CheckRevocationStatus(); err != nil { + return fmt.Errorf("revocation check failed: %w", err) + } + + return nil +} +``` + +Security Checklist: +1. All DIDs follow W3C specification +2. Credentials implement proper proofs +3. Keys use proper derivation/rotation +4. State changes are validated +5. Access control is enforced +6. Data is properly encrypted +7. Logging captures security events + +Refer to W3C specifications, Cosmos SDK security documentation, and blockchain security best practices for detailed implementation guidance. diff --git a/README.md b/README.md index 051a40d..8500103 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,8 @@ -# CosmES -[![npm version](https://badge.fury.io/js/cosmes.svg)](https://www.npmjs.com/package/cosmes) +# SonrES + +[![npm version](https://badge.fury.io/js/cosmes.svg)](https://www.npmjs.com/package/@onsonr/es) A tree-shakeable, framework agnostic, [pure ESM](https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c) alternative of [CosmJS](https://github.com/cosmos/cosmjs) and [Cosmos Kit](https://cosmoskit.com) (**generate bundles up to 10x smaller than Cosmos Kit**). @@ -28,7 +29,7 @@ A tree-shakeable, framework agnostic, [pure ESM](https://gist.github.com/sindres - **Fully tree-shakeable**: import and bundle only the modules you need - **Framework agnostic**: integrate with any web framework (React, Vue, Svelte, Solid, etc.) -- **Lightweight and minimal**: 153 KB gzipped to connect a React app to Keplr via browser extension or WalletConnect, 10x smaller than Cosmos Kit V2 (see [benchmarks](#benchmarks)) +- **Lightweight and minimal**: 153 KB gzipped to connect a React app to Keplr via browser extension or WalletConnect, 10x smaller than Cosmos Kit V2 (see [benchmarks](#benchmarks)) - **Uses modern web APIs**: no dependencies on Node.js and minimal dependencies on third-party libraries where possible - **Supports modern bundlers**: works with Vite, SWC, Rollup, etc. - **Fully typed**: written in TypeScript and ships with type definitions @@ -68,7 +69,7 @@ This library only exports ES modules. To ensure imports from this library work c { "compilerOptions": { "moduleResolution": "bundler", // recommended if using modern bundlers - // or "node16" + // or "node16" // or "nodenext" // but NOT "node" } @@ -162,7 +163,7 @@ This directory is a [Cosmos Kit](https://cosmoskit.com) alternative to interact ## Benchmarks -See the [`benchmarks`](./benchmarks) folder, where the bundle size of CosmES is compared against Cosmos Kit. The following are adhered to: +See the [`benchmarks`](./benchmarks) folder, where the bundle size of SonrES is compared against Cosmos Kit. The following are adhered to: - Apps should only contain the minimal functionality of connecting to Osmosis via Keplr using both the browser extension and WalletConnect wallets - Apps should be built using React 18 (as Cosmos Kit has a [hard dependency](https://docs.cosmoskit.com/get-started)) and Vite @@ -173,8 +174,8 @@ See the [`benchmarks`](./benchmarks) folder, where the bundle size of CosmES is > Last updated: 4th May 2024 | Package | Minified | Gzipped | -|---------------|----------|---------| -| CosmES | 553 KB | 153 KB | +| ------------- | -------- | ------- | +| SonrES | 553 KB | 153 KB | | Cosmos Kit v1 | 6010 KB | 1399 KB | | Cosmos Kit v2 | 6780 KB | 1556 KB | diff --git a/package.json b/package.json index 16a074f..edfc637 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@onsonr/es", - "version": "0.5.3", + "version": "0.0.1", "private": false, "packageManager": "pnpm@8.3.0", "sideEffects": false, @@ -12,6 +12,7 @@ "src", "dist" ], + "jsdelivr": "dist/client/index.js", "exports": { "./client": "./dist/client/index.js", "./codec": "./dist/codec/index.js",