Description to implement plugins using wasmtime

- Provide Proposals to implement parser & source & composing them
  together and a stand-alone producer

developer/src/SUMMARY.md

@@ -10,3 +10,7 @@
         - [Advantages](plugin/wasmtime/advantages.md)
         - [Disadvantages](plugin/wasmtime/disadvantages.md)
         - [Notes & Tips](plugin/wasmtime/notes-tips.md)
+        - [Plugin Proposal](plugin/wasmtime/proposal.md) 
+            - [Parser](plugin/wasmtime/proposal-parser.md) 
+            - [Source](plugin/wasmtime/proposal-source.md) 
+            - [Producer & Composing](plugin/wasmtime/proposal-producer.md) 

developer/src/plugin/wasmtime/proposal-parser.md

@@ -0,0 +1,60 @@
+# Parser
+
+* Data types for transfer are define in `parsing` interface within the `wit` file, alongside functions provided by the host to retrieve parse results.
+* The resource `parser` is encapsulated within `parse-client` interface, providing methods for creation, configuration provisioning, and parse invocation.
+* The host serves as a wrapper over the plugin, implementing the parser trait to integrate with the host while abstracting plugin details
+* Parse calls process the entire given bytes buffer, returning a list of parse results, which will be cached on the host for internal system integration.
+* Changing the parser signature to return an iterator of results would improve the code. 
+* DLT-Parser as plugin provided performance that is about 1.35x compared to the native (plugin is 33% slower than native). File size: 500 MB, native: 10.3 Seconds, Plugin: 13.6 seconds.
+* An alternative parser is implemented that keeps track on the current memory state so avoid unnecessary data copying, providing one result per call. However, it had worse performance due to the overhead of repeated plugin function calls on each `parse()` call. 
+
+## WIT file:
+```wit, ignore
+
+interface parsing {
+  // *** Data types ***
+  record parse-return {
+    value: option<parse-yield>,
+    cursor: u64,
+  }
+
+  record attachment {...}
+
+  variant parse-yield {
+    message(string),
+    attachment(attachment),
+    message-and-attachment(tuple<string, attachment>),
+  }
+
+  variant error {...}
+
+  // Host functions that can be called from the plugin to provide the result
+  add: func(item: result<parse-return, error>);
+  add-range: func(items: list<result<parse-return, error>>);
+}
+
+interface parse-client {
+  use parsing.{error, parse-return};
+
+  // Parser trait definitions
+  resource parser {
+    // Create and initialize the plugin, giving it the path for its configurations
+    constructor();
+    init: func(config-path: stirng) -> result<_,error>;
+
+    // *** Parse functions *** 
+    // Returns all the results as list to the host without using host methods
+    parse: func(data: list<u8>, timestamp: option<u64>) -> list<result<parse-return, error>>;
+
+    // Use host methods to return the results (Save memory allocation for results list on plugin)
+    parse-res: func(data: list<u8>, timestamp: option<u64>);
+  }
+}
+
+// State that the plugin must provide parser implementation in this world.
+world parse {
+  export parse-client;
+}
+
+```
+

developer/src/plugin/wasmtime/proposal-producer.md

@@ -0,0 +1,46 @@
+# Stand-alone Producer & Parser and Source Composition
+
+* The component model provides tools to compose many plugins together, resolving their dependencies to produce one plugin with combined from capabilities.
+* With the approach we can compose the parser and the source plugins with an additional buffering plugin to produce a unified plugin that accepts read and parse configuration as input then it reads the data, process them returning results to the host.
+* In this scenario, the host must replace the whole producer functionality in the native code, as all processing logic is encapsulated within the plugin.
+* The same `wit` API and host implementation can be utilized with stand-alone producers too which can be useful for `MDF` file format.
+
+## Current Prototype:
+
+* For prototyping purposes, a plugin has been provided that reads the source and offers basic buffering functionality. It then calls a method to parse the data after it has been read and buffered..
+* After compiling the plugin, it can be composed  with any parser plugin using [WebAssembly Compositions (WAC)](https://github.com/bytecodealliance/wac) CLI tool to produce one producer plugin.
+* Composing command: 
+```bash, ignore
+wac plug --plug {path_to_parser_wasm_file} -o {output_file.wasm} {path_to_source_producer_wasm_file}
+```
+* On the host side the plugin wrapper provides similar functions like the producer, which is a function to provide stream of parse results.
+* Replacing Producer with this Prototype for DLT files results in performance that is about 1.32x than the native one (30% slower).
+* The current implementation is just for prototyping purposes here and stills far away from the real-world implementation.
+
+## WIT File
+
+```wit, ignore
+// *** Data Types & Traits Definitions ***
+interface parsing { ... }
+interface parse-client { ... }
+interface sourcing { ... }
+
+interface source-prod-client {
+  use sourcing.{source-error};
+  use parsing.{};
+  use parse-client.{parser};
+
+  // Trait that initialize a parser source, reads and buffers the data,
+  // then call parse on them.
+  resource source-prod{
+    constructor();
+    init: func(config-path: string, file-path: string) -> result<_, source-error>;
+    read-then-parse: func(len: u64, bytes-read: u64, timestamp: option<u64>) -> result<_, source-error>;
+  }
+}
+
+// State that the plugin must provide `soruce-prod` implementation in this world
+world producer {
+  export source-prod-client;
+}
+```

developer/src/plugin/wasmtime/proposal-source.md

@@ -0,0 +1,31 @@
+# Source
+
+* Current implementation covers the read part of the Byte-Source only, having the plugin to provide the method `read()` returning an array of bytes only.
+* Acting as a wrapper, the host implements the read trait and is provided as an argument to the native `BinaryByteSource` Struct.
+* With this simple approach it's possible to achieve a near-native performance with the plugin being only 2% slower than native. This is primarily due to all buffering being handled within the native part.
+
+## WIT File
+
+```wit, ignore
+interface sourcing {
+  // *** Data Types ***
+  variant source-error {...}
+}
+
+interface source-client {
+  use sourcing.{source-error};
+
+  // Client trait definition 
+  resource byte-source{
+    constructor();
+    init: func(config-path: string, file-path: string) -> result<_, source-error>;
+    read: func(len: u64) -> result<list<u8>, source-error>;
+  }
+}
+
+// State that the plugin must provide source implementation in this world.
+world source {
+  export source-client;
+}
+```
+

developer/src/plugin/wasmtime/proposal.md

@@ -0,0 +1,6 @@
+# Plugin Proposal
+
+* The General Proposal is to use `wit` files to define the data types and functions provided by both the host and the plugins.
+* Using `wit` files allows for  plugins written in any language that can compile to WebAssembly, provided they comply to the contract defined in the WIT file.
+* For Rust plugin we can provide project templates (can be used with [cargo generate](https://github.com/cargo-generate/cargo-generate)) and code examples. Additionally, we can provide a crate containing macros or functions to help the users in implementing the necessary functionality only.
+