Merge pull request #567 from streamingfast/giu/updates

Implementing Docs Feedback

docs/SUMMARY.md

@@ -45,7 +45,12 @@
   * [Substreams:Stream](new/how-to-guides/sinks/stream/stream.md)
     * [JavaScript](new/how-to-guides/sinks/stream/javascript.md)
     * [Go](new/how-to-guides/sinks/stream/go.md)    
-  * [PubSub](new/how-to-guides/sinks/pubsub.md)
+  * [Substreams:PubSub](new/how-to-guides/sinks/pubsub.md)
+  * [Community Sinks](new/how-to-guides/sinks/community/other-sinks)
+    * [MongoDB](new/how-to-guides/sinks/community/other-sinks/mongodb.md)
+    * [Files](new/how-to-guides/sinks/community/other-sinks/files.md)
+    * [Key-Value Store](new/how-to-guides/sinks/community/other-sinks/kv.md)
+    * [Prometheus](new/how-to-guides/sinks/community/other-sinks/prometheus.md)
 <!--* [Deploying Substreams](new/tutorials/deployable-services/deployable-services.md)-->
 * [EVM Extensions](new/how-to-guides/eth-calls/overview.md)
   * [Making eth\_calls](new/how-to-guides/eth-calls/eth-calls.md)
@@ -77,15 +82,10 @@
     * [Aggregation Windows](new/references/substreams-components/modules/aggregation-windows.md)
     * [Parameterized Modules](new/references/substreams-components/modules/parameterized-modules.md)
   * [Manifests Reference](new/references/substreams-components/manifests.md)
-* [Other Ways of Consuming](new/references/community/other-sinks/README.md)
-    * [Files](new/references/community/other-sinks/files.md)
-    * [Key-Value Store](new/references/community/other-sinks/kv.md)
-    * [MongoDB](new/references/community/other-sinks/mongodb.md)
-    * [Prometheus](new/references/community/other-sinks/prometheus.md)
 * [Substreams Architecture](new//references/architecture.md)
-* [Graph-Node](new/tutorials/graph-node/graph-node.md)
-  * [Local Development](new/tutorials/graph-node/local-development.md)
-  * [Publish to The Graph Network](new/tutorials/graph-node/publish-decentralized-network.md)
+* [Graph-Node](./new/references/graph-node/graph-node.md)
+  * [Local Development](./new/references/graph-node/local-development.md)
+  * [Publish to The Graph Network](./new/references/graph-node/publish-decentralized-network.md)
 * [Indexer Reference](new/references/indexer-reference/indexer-reference.md)
   * [Test Substreams Locally](new/references/indexer-reference/test-locally.md)
 <!-- * [Glossary](new/references/glossary.md) -->

docs/new/how-to-guides/develop-your-own-substreams/evm/exploring-ethereum/map_block_meta_module.md

@@ -2,12 +2,6 @@
 
 This module takes a raw Ethereum block and returns a reduced version of a block, with just three pieces of information: hash, parent hash, and block number.
 
-{% hint style="success" %}
-**Tip**: This tutorial teaches you how to build a Substreams from scratch.
-
-Remember that you can auto-generate a filtered Substreams module by using the [code-generation tools](../../../../tutorials/evm.md).
-{% endhint %}
-
 Let's run the Substreams first, and then go through the code.
 
 ### Running the Substreams

docs/new/how-to-guides/develop-your-own-substreams/evm/exploring-ethereum/map_filter_transactions_module.md

@@ -3,12 +3,6 @@
 This module iterates over all the blockchain transactions and filters them by some of their fields (the `from` and `to` fields).
 For example, if you want to retrieve all the transactions initiated by the address `0xb6692f7ae54e89da0269c1bfd685ccdfd41d2bf7`, you set the filter `from = 0xb6692f7ae54e89da0269c1bfd685ccdfd41d2bf7`.
 
-{% hint style="success" %}
-**Tip**: This tutorial teaches you how to build a Substreams from scratch.
-
-Remember that you can auto-generate your Substreams module by usig the [code-generation tools](../../../getting-started/evm/emv-first-sql.md).
-{% endhint %}
-
 ### Running the Substreams
 
 First, generate the Protobuf modules and build the Rust code:

docs/new/how-to-guides/sinks/sinks.md

@@ -1,4 +1,42 @@
-Once you find a package that fits your needs, you can choose how you want to consume the data:
+Once you find a package that fits your needs, you can choose how you want to consume the data. Sinks are integrations that allow you to send the extracted data to different destinations, such as a SQL database, a file or a subgraph. 
+
+{% hint style="info" %}
+**Note**: Some of the sinks are officially supported by StreamingFast (i.e. active support is provided), but other sinks are community-driven and support can't be guaranteed. 
+{% endhint %}
+
 - [SQL Database](./sql/sql-sink.md): Send the data to a database.
 - [Subgraph](./subgraph/subgraph.md): Configure an API to meet your data needs and host it on The Graph Network.
-- [Direct Streaming](./stream/stream.md): Stream data directly from your application.
+- [Direct Streaming](./stream/stream.md): Stream data directly from your application.
+- [Community Sinks](new/how-to-guides/sinks/community/other-sinks): Explore quality community maintained sinks. 
+
+{% hint style="success" %}
+**Deployable Service**: If you’d like your sink (e.g., SQL or PubSub) to be hosted for you, reach out to the StreamingFast team [here](mailto:[email protected]). 
+{% endhint %}
+
+## Navigating Sink Repos
+
+### Official
+
+| Name      | Support | Maintainer       | Source Code |
+|-----------|---------|------------------|-------------|
+| SQL       | O       | StreamingFast    |[substreams-sink-sql](https://github.com/streamingfast/substreams-sink-sql)|
+| Go SDK    | O       | StreamingFast    |[substreams-sink](https://github.com/streamingfast/substreams-sink)|
+| Rust SDK  | O       | StreamingFast    |[substreams-sink-rust](https://github.com/streamingfast/substreams-sink-rust)|
+| JS SDK    | O       | StreamingFast    |[substreams-js](https://github.com/substreams-js/substreams-js)|
+| KV Store  | O       | StreamingFast    |[substreams-sink-kv](https://github.com/streamingfast/substreams-sink-kv)|
+| Prometheus| O       | Pinax            |[substreams-sink-prometheus](https://github.com/pinax-network/substreams-sink-prometheus)|
+| Webhook   | O       | Pinax            |[substreams-sink-webhook](https://github.com/pinax-network/substreams-sink-webhook)|
+| CSV       | O       | Pinax            |[substreams-sink-csv](https://github.com/pinax-network/substreams-sink-csv)|
+| PubSub    | O       | StreamingFast    |[substreams-sink-pubsub](https://github.com/streamingfast/substreams-sink-pubsub)|
+
+### Community
+
+| Name      | Support | Maintainer       | Source Code |
+|-----------|---------|------------------|-------------|
+| MongoDB   | C       | Community        |[substreams-sink-mongodb](https://github.com/streamingfast/substreams-sink-mongodb)|
+| Files     | C       | Community        |[substreams-sink-files](https://github.com/streamingfast/substreams-sink-files)|
+| KV Store  | C       | Community        |[substreams-sink-kv](https://github.com/streamingfast/substreams-sink-kv)|
+| Prometheus| C       | Community        |[substreams-sink-Prometheus](https://github.com/pinax-network/substreams-sink-prometheus)|
+
+* O = Official Support (by one of the main Substreams providers)
+* C = Community Support

docs/new/how-to-guides/sinks/subgraph/subgraph.md

@@ -1,14 +1,17 @@
 It is possible to the send data the of a Substreams to a subgraph, thus creating a Substreams-powered Subgraph.
 
-There are two ways of making Substreams interact with subgraphs:
+There are two ways of making Substreams sink to a subgraph:
 - Create a special [graph_out module](./graph-out.md) that emits an [EntityChanges](https://github.com/streamingfast/substreams-sink-entity-changes/blob/develop/proto/sf/substreams/sink/entity/v1/entity.proto#L11) Protobuf.
 The subgraph will read the `EntityChanges` object and consume the data.
 - Use the [**Substreams triggers**](./triggers.md) to consume Substreams Protobuf directly inside your subgraph.
 
 ## What Option To Use
-Both are valid options depending on the use case:
-- Use the `graph_out` module is if your logic is entirely in the Substreams Rust code. You will create the subgraph entities in the Substreams and the subgraph will only read them. By putting your logic in the Substreams, you will **benefit from parallelism and you will be able to handle a cursor to track the progress of the data**. Read more about it in the [Never Miss Data](../../../references/reliability-guarantees.md) section. 
+It is really a matter of where you put your logic, in the subgraph or the Substreams.
 
-- Use the **Substreams triggers** if you want to use Substreams as an extraction layer for the data, but the logic of your application will be in the subgraph AssemblyScript code. By putting all your logic in the subgraph, you will deal with an _easier_ programming language (AssemblyScript).
+- [Substreams Triggers](./triggers.md): Consume from any Substreams module by importing the Protobuf model through a subgraph handler and write all your transformations ussing AssemblyScript. This method creates the subgraph entities directly in the subgraph.
+- [Substreams Graph-Out](./graph-out.md): By writing more of the logic into Substreams, you can consume the module's output directly into `graph-node`. You will create the subgraph entities in the Substreams and the subgraph will read them. 
+
+Having more of your logic in Substreams benefits from a parallelized model and a cursor to [never miss data](../../../references/reliability-guarantees.md), whereas triggers will be linearly consumed in `graph-node`.
+
 
 <figure><img src="../../../.gitbook/assets/consume/service-subgraph.png" width="100%" /></figure>

docs/new/tutorials/cosmos-compatible/injective.md

@@ -7,7 +7,7 @@ In this guide, you'll learn how to initialize a Injective-based Substreams proje
 
 ## Step 1: Initialize Your Injective Substreams Project
 
-1. Open the [Dev Container](https://github.com/streamingfast/substreams-starter) and follow the on-screen steps or `README.md` to initialize your project.
+1. Open the [Dev Container](https://github.com/streamingfast/substreams-starter) and follow the on-screen steps to initialize your project.
 
 2. Running `substreams init` will give you the option to choose between two Injective project options. Select the one that best fits your requirements:
     - **Injective-minimal**: Creates a simple Substreams that extracts raw Injective block data and generates corresponding Rust code. This path will start you with the full raw block, you can navigate to the `substreams.yaml` (the manifest) to modify the input.
@@ -19,7 +19,7 @@ Tip: Have the start block of your transaction or specific events ready.
 
 ## Step 2: Visualize the Data
 
-1. Running `substreams auth` will prompt you to create your account [here](https://thegraph.market/) to generate an authentification token (JWT), pass it back as input.
+1. Run `substreams auth` to create your [account](https://thegraph.market/) and generate an authentification token (JWT), then pass this token back as input.
 
 2. Now you can freely use the `substreams gui` to visualize and itterate on your extracted data.
 
@@ -29,7 +29,23 @@ Within the generated directories, modify your Substreams modules to include addi
 
 ## Step 3: Load the Data
 
-To make your Substreams queriable (as opposed to [direct streaming](../how-to-guides/sinks/stream/stream.md)), you can automatically generate a Subgraph (known as a [Substreams-powered subgraph](https://thegraph.com/docs/en/sps/introduction/)) or SQL-DB sink by following the on-screen steps or referring to the `README.md`. 
+To make your Substreams queriable (as opposed to [direct streaming](../how-to-guides/sinks/stream/stream.md)), you can automatically generate a Subgraph (known as a [Substreams-powered subgraph](https://thegraph.com/docs/en/sps/introduction/)) or SQL-DB sink.
+
+### Subgraph
+
+1. Run `substreams codegen subgraph` to intialize the sink, producing the neccessary files and function definitions. 
+2. Create your [subgraph mappings](../how-to-guides/sinks/subgraph/triggers.md) within the `mappings.ts` and associated entities within the `schema.graphql`.
+3.  Deploy
+
+### SQL
+
+1. Run `substreams codegen sql` and choose from either ClickHouse or Postgres to intialize the sink, producing the neccessary files. 
+2. Run `substreams build` build the [Substreams:SQL](../how-to-guides/sinks/sql/sql-sink.md) sink. 
+3. Run `substreams-sink-sql` to sink the data into your selected SQL DB.
+
+{% hint style="info" %}
+**Note**: Run `help` to better navigate the development environment and check the health of containers. 
+{% endhint %}
 
 ## Additional Resources
 

docs/new/tutorials/cosmos-compatible/mantra.md

@@ -2,7 +2,7 @@ In this guide, you'll learn how to initialize a MANTRA-based Substreams project
 
 ## Step 1: Initialize Your MANTRA Substreams Project
 
-1. Open the [Dev Container](https://github.com/streamingfast/substreams-starter) and follow the on-screen steps or `README.md` to initialize your project.
+1. Open the [Dev Container](https://github.com/streamingfast/substreams-starter) and follow the on-screen steps to initialize your project.
 
 2. Running `substreams init` will give you the option to choose between two MANTRA project options. Select the one that best fits your requirements:
     - **MANTRA-minimal**: Creates a simple Substreams that extracts raw MANTRA block data and generates corresponding Rust code. This path will start you with the full raw block, you can navigate to the `substreams.yaml` (the manifest) to modify the input.
@@ -14,7 +14,7 @@ Tip: Have the start block of your transaction or specific events ready.
 
 ## Step 2: Visualize the Data
 
-1. Running `substreams auth` will prompt you to create your account [here](https://thegraph.market/) to generate an authentification token (JWT), pass it back as input.
+1. Run `substreams auth` to create your [account](https://thegraph.market/) and generate an authentification token (JWT), then pass this token back as input.
 
 2. Now you can freely use the `substreams gui` to visualize and itterate on your extracted data.
 
@@ -24,7 +24,23 @@ Within the generated directories, modify your Substreams modules to include addi
 
 ## Step 3: Load the Data
 
-To make your Substreams queriable (as opposed to [direct streaming](../how-to-guides/sinks/stream/stream.md)), you can automatically generate a Subgraph (known as a [Substreams-powered subgraph](https://thegraph.com/docs/en/sps/introduction/)) or SQL-DB sink by following the on-screen steps or referring to the `README.md`. 
+To make your Substreams queriable (as opposed to [direct streaming](../how-to-guides/sinks/stream/stream.md)), you can automatically generate a Subgraph (known as a [Substreams-powered subgraph](https://thegraph.com/docs/en/sps/introduction/)) or SQL-DB sink.
+
+### Subgraph
+
+1. Run `substreams codegen subgraph` to intialize the sink, producing the neccessary files and function definitions. 
+2. Create your [subgraph mappings](../how-to-guides/sinks/subgraph/triggers.md) within the `mappings.ts` and associated entities within the `schema.graphql`.
+3.  Deploy
+
+### SQL
+
+1. Run `substreams codegen sql` and choose from either ClickHouse or Postgres to intialize the sink, producing the neccessary files. 
+2. Run `substreams build` build the [Substreams:SQL](../how-to-guides/sinks/sql/sql-sink.md) sink. 
+3. Run `substreams-sink-sql` to sink the data into your selected SQL DB.
+
+{% hint style="info" %}
+**Note**: Run `help` to better navigate the development environment and check the health of containers. 
+{% endhint %}
 
 ## Additional Resources