Merge pull request #3 from czelabueno/feature/rearchicture-project

rearchitecture of LC4J-workflow for 2025 features
czelabueno · Dec 29, 2024 · 17cb9b4 · 17cb9b4
2 parents 2f3d0c0 + 2af552d
commit 17cb9b4
Show file tree

Hide file tree

Showing 29 changed files with 1,360 additions and 267 deletions.
diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml
@@ -0,0 +1,41 @@
+name: jAI Workflow CI
+
+on:
+  push:
+      branches:
+      - main
+  pull_request:
+      branches:
+      - main
+
+jobs:
+  java_build:
+    strategy:
+      matrix:
+          java_version: ['17', '21']
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Set up JDK ${{ matrix.java_version }}
+        uses: actions/setup-java@v4
+        with:
+          java-version: ${{ matrix.java_version }}
+          distribution: 'temurin'
+          cache: maven
+      - name: Build and test with JDK ${{ matrix.java_version }}
+        run: |
+          mvn -B -U -T4C clean test \
+            -Dmaven.javadoc.skip=true surefire-report:report
+      - name: Upload Surefire Reports with JDK ${{ matrix.java_version }}
+        uses: actions/upload-artifact@v4
+        with:
+          name: test-surefire-reports-jdk-${{ matrix.java_version }}
+          path: '**/target/surefire-reports/*.xml'
+      - name: Publish Test Report
+        uses: mikepenz/action-junit-report@v4
+        if: success() || failure() # always run even if the previous step fails
+        with:
+          report_paths: '**/target/*-reports/TEST-*.xml'
+          annotate_only: true
+
+
diff --git a/.github/workflows/release.yaml b/.github/workflows/release.yaml
@@ -0,0 +1,29 @@
+name: jAI Workflow Release
+
+on:
+  workflow_dispatch:
+
+jobs:
+  release:
+    runs-on: ubuntu-latest
+    steps:
+        - uses: actions/checkout@v4
+
+        - name: Set up JDK 21
+          uses: actions/setup-java@v4
+          with:
+             java-version: '21'
+             distribution: 'temurin'
+             cache: maven
+             server-id: ossrh
+             server-username: OSSRH_USERNAME
+             server-password: OSSRH_PASSWORD
+             gpg-private-key: ${{ secrets.GPG_PRIVATE_KEY }}
+             gpg-passphrase: GPG_PASSPHRASE
+
+        - name: release
+          run: mvn -B -U -Prelease clean deploy -DskipTests
+          env:
+             GPG_PASSPHRASE: ${{ secrets.GPG_PASSPHRASE }}
+             OSSRH_USERNAME: ${{ secrets.OSSRH_USERNAME }}
+             OSSRH_PASSWORD: ${{ secrets.OSSRH_PASSWORD }}
diff --git a/README.md b/README.md
@@ -1,40 +1,94 @@
-# LangChain4j Workflow 🦜🔀
+# JavAI Workflow 🦜🔀: Build programmatically custom agentic workflows, AI Agents, RAG systems for java
+[![Build Status](https://github.com/czelabueno/langchain4j-workflow/actions/workflows/ci.yaml/badge.svg)](https://github.com/czelabueno/langchain4j-workflow/actions/workflows/ci.yaml)
+[![Maven Central](https://maven-badges.herokuapp.com/maven-central/dev.langchain4j/langchain4j-workflow/badge.svg)](https://maven-badges.herokuapp.com/maven-central/dev.langchain4j/langchain4j-workflow)
+[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
 
-Build advanced java applications with AI based on flexible stateful workflows 💡
+An open-source Java library to build, package, integrate, orchestrate and monitor agentic AI systems for java developers 💡
 
-## Overview
-LangChain4j Workflow is a dynamic, stateful workflow engine crafted as a Java library, drawing inspiration from graph network libraries. It empowers developers with granular control over the flow and state of their applications. This engine is a game-changer for building sophisticated AI applications, such as RAG-based approaches using modern paradigms and agent architectures, where the application's flow and state are pivotal. It enables the crafting of custom behavior, leading to a significant reduction in hallucinations and an increase in response reliability
-
-LangChain4j Workflow is influenced by [LangGraph](https://langchain-ai.github.io/langgraph/tutorials/introduction/), [Graphviz](https://graphviz.gitlab.io/Gallery/directed/) and [Apache Beam](https://beam.apache.org/), and it offers a multitude of benefits. It allows you to define custom workflows as a graph, iteratively, with cycles, flexibility, control, and conditional decisions. These benefits are indispensable for building advanced AI applications.
+![Workflow Image](docs/jai-worflow-anatomy.png)
 
-LangChain4j Workflow is designed to integrate seamlessly with [LangChain4j](https://docs.langchain4j.dev/), enabling you to define custom workflows using all the features that LangChain4j offers. This integration could provide a comprehensive toolset for building advanced AI applications.
+> 🌟 **Starring me**: If you find this repository beneficial, don't forget to give it a star! 🌟 It's a simple way to show your appreciation and help this project grow!
 
-> 🌟 **Starring me**: If you find this repository beneficial, don't forget to give it a star! 🌟 Your support increases its chances of being merged with the LangChain4j codebase. It's a simple way to show your appreciation and help this project grow!
-
-## Key Features
-- **Stateful**: LangChain4j Workflow is a stateful engine, enabling you to design custom states as POJO and transitions. This feature provides a robust foundation for managing the flow and state of your application.
-- **Graph-Based**: The workflow is graph-based, offering the flexibility to define custom workflows with multiple directions such as one-way, round trip, cyclic, and more. This feature allows for intricate control over the flow of your application.
-- **Flexible**: LangChain4j Workflow is designed with flexibility in mind. You can define custom workflows and append them at any point in other RAG paradigms, such as Modular RAG. This flexibility allows for a high degree of customization.
-- **Iterative**: The engine supports the implementation of loops and conditionals in your custom workflows. This feature allows for complex logic and flow control within your workflows.
-- **Streaming Support**: LangChain4j Workflow supports streaming outputs as they are produced by each node. This feature allows for real-time processing and response in your application.
-- **Integration**: LangChain4j Workflow is designed to integrate seamlessly with [LangChain4j](https://docs.langchain4j.dev/), enabling you to define custom workflows using all the features that LangChain4j offers. This integration provides a comprehensive toolset for building advanced AI applications.
+## Overview
+JavAI Workflow (named initially Langchain4j-workflow) is a dynamic, stateful workflow engine crafted as a Java library. It empowers java developers with granular control over the orchestrated workflows as a graph, iteratively, with cycles, flexibility, control, and conditional decisions. This engine is a game-changer for building sophisticated AI applications, such as multiples RAG-based approaches using modern paradigms and agent architectures. It enables the crafting of custom behavior, leading to a significant reduction in hallucinations and an increase in response reliability.
+
+jAI Workflow is influenced by [LangFlow](https://github.com/langflow-ai/langflow), [LangGraph](https://langchain-ai.github.io/langgraph/tutorials/introduction/), [Graphviz](https://graphviz.gitlab.io/Gallery/directed/).
+
+## Principles:
+- **Java-based** jAI workflows are configuration as code (java) and agnostic to AI models, enabling you can define custom advanced and dynamic workflows just writing Java code.
+- **Stateful**: jAI Workflow is a stateful engine, enabling you to design custom states as POJO and transitions. This feature provides a robust foundation for managing the flow and state of your application.
+- **Graph-Based**: The workflow is graph-based, offering the flexibility to define custom workflows with multiple directions such as one-way, round trip, loop, recursive and more. This feature allows for intricate control over the flow of your application.
+- **Flexible**: jAI Workflow is designed with flexibility in mind. You can define custom workflows, modules or agents to build RAG systems as LEGO-like. A module can be decoupled and integrated in any other workflow.
+- **Ecosystem integration**: jAI workflow will be integrated with any java AI project. It provides a comprehensive toolset for building advanced java AI applications.
+- **Publish as API**: jAI Workflow can be published as an API as entrypoint once you have defined your custom workflow. This feature allows you to expose your workflow as a service.
+- **Observability**: jAI Workflow provides observability features to monitor the execution of the workflow, trace inputs and outputs, and debug the flow of your application.
+- **Scalable**: jAI Workflow can be deployed as any java project as standalone or distributed mode as containers in any cloud provider or kubernetes environment. Each module can run in a different JVM env or container for scalability in production environments.
+
+## 🚀 Features
+### v.0.2.0 Features
+- **Graph-core**: The engine supports create `Nodes`, `Conditional Nodes`, `Edges`, and workflows as a graph. This feature allows you to define custom workflows with multiple `Transitions` between nodes such as one-way, round trip and recursive. 
+- **Run workflow**: jAI Workflow supports synchronized `workflow.run()` and streaming `worflow.runStream()` runs the outputs as they are produced by each node. This last feature allows for real-time processing and response in your application.
+- **Integration**: [LangChain4j](https://docs.langchain4j.dev/) integration, enabling you to define custom workflows using all the features that LangChain4j offers. This integration provides a comprehensive toolset for building advanced AI applications to integrate with multiple LLM providers and models.
 - **Visualization**: The engine supports the generation of workflow images. This feature allows you to visualize the flow computed of your app workflow. By Default it uses `Graphhviz` lib to generate the image, but you implement your own image generator on `GraphImageGenerator.java` interface.
-
-## Installation
-```shell
-mvn clean package install
+### Q1 2025 Features
+- **Graph-Core**:
+  - Split Nodes
+  - Merge Nodes
+  - Parallel transitions
+  - Human-in-the-loop
+- **Modular (Group of nodes)**:
+  - Module
+  - Remote Module
+- **Integration**:
+  - Model Context Protocol (MCP) integration as server and client.
+  - Define remote module as MCP server.
+- **Observability**:
+  - OpenTelemetry integration (metrics and traces).
+  - Debugging mode logging structure.
+### 🗺️ Future Features
+- **Deployment Model**:
+  - Dockerize workflow
+  - Kubernetes deployment
+  - Cloud deployment
+- **API**:
+  - Publish workflow as API (SSE for streaming runs and REST for sync runs).
+- **Playground**:
+  - Web-based playground to add, test and run jAI workflows APIs.
+  - Chatbot Q&A viewer.
+  - Graph tracing visualization for debugging
+
+## Architecture
+jAI Workflow is designed with a modular architecture, enabling you to define custom workflows, modules, or agents to build RAG systems as LEGO-like. A module can be decoupled and integrated into any other workflow.
+
+### Standalone Architecture
+![Standalone Architecture](docs/jai-standalone-architecture.png)
+
+### Distributed Architecture
+![Desired Architecture](docs/jai-distributed-architecture.png)
+
+> 📖 Full documentation will be available soon
+
+## 💡How to use in your Java project
+In **jAI Workflow**, the notion of state plays a pivotal role. Every execution of the graph initiates a state, which is then transferred among the nodes during their execution. Each node, after its execution, updates this internal state with its own return value. The method by which the graph updates its internal state is determined by user-defined functions.
+
+The simplest way to use jAI Workflow in your project is with the [LangChain4j](https://docs.langchain4j.dev) integration because enables you to define custom workflows using all the features that LangChain4j offers. This integration could provide a comprehensive toolset for building advanced AI applications:
+```xml
+<dependency>
+  <groupId>com.github.czelabueno</groupId>
+  <artifactId>langchain4j-workflow</artifactId>
+  <version>0.2.0</version> <!--Change to the latest version-->
+</dependency>
 ```
-## Example
-In **LangChain4j Workflow**, the notion of state plays a pivotal role. Every execution of the graph initiates a state, which is then transferred among the nodes during their execution. Each node, after its execution, updates this internal state with its own return value. The method by which the graph updates its internal state is determined by user-defined functions.
 
-Add the following dependency to your `pom.xml` file:
+If you would want to use jAI workflow without LangChain4j or with other framework, add the following dependency to your `pom.xml` file:
 ```xml
 <dependency>
-  <groupId>dev.langchain4j</groupId>
-  <artifactId>langchain4j-workflow</artifactId>
-  <version>0.1.0</version> <!--Change to the latest version-->
+  <groupId>com.github.czelabueno</groupId>
+  <artifactId>jai-workflow-core</artifactId>
+  <version>0.2.0</version> <!--Change to the latest version-->
 </dependency>
 ```
+### Example
 Define a stateful bean with fields that will be used to store the state of the workflow:
 ```java
 // Define a stateful bean
@@ -150,11 +204,17 @@ You can print all computed transitions:
 START -> node1 -> node2 -> node3 -> node2 -> node3 -> node4 -> END
 ```
 You can generate a workflow image with all computed transitions:
-
-![Workflow Image](image/my-workflow.svg)
+```shell
+> image/
+> ├── my-workflow.svg
+```
+![Workflow Image](jai-workflow-core/image/my-workflow.svg)
 
 ## LLM examples
-You can check all examples in the [langchain4j-worflow-examples](https://github.com/czelabueno/langchain4j-workflow-examples) repository. Please note that examples can be modified and more examples will be added over time.
+You can check all examples in the [langchain4j-worflow-examples](https://github.com/czelabueno/langchain4j-workflow-examples) repository where show you how-to implement multiple RAG patterns, agent architectures and AI papers using LangChain4j and jAI Workflow. 
+
+> Please note that examples can be modified and more examples will be added over time.
+
 ### MoA
 - **Mixture-of-Agents (MoA)**:
   - Java example: [`langchain4j-moa`](https://github.com/czelabueno/langchain4j-workflow-examples/tree/main/langchain4j-moa)
@@ -185,9 +245,9 @@ You can check all examples in the [langchain4j-worflow-examples](https://github.
   - Java example: _Very soon_
   - Based on Paper: https://arxiv.org/pdf/2305.04091
 
-## Contribute & feedback
+## 💬 Contribute & feedback
 If you have any feedback, suggestions, or want to contribute, please feel free to open an issue or a pull request. We are open to new ideas and suggestions.
-Help us to maturity this project and make it more useful for the community in order to merge it with LangChain4j source code.
+Help us to maturity this project and make it more useful for the java community.
 
-## Authors
-- Carlos Zela [@c_zela](https://x.com/c_zela)
+## 🧑🏻‍💻 Authors
+- Carlos Zela [@c_zela](https://x.com/c_zela) [czelabueno](https://linkedin.com/in/czelabueno)
diff --git a/docs/jai-distributed-architecture.png b/docs/jai-distributed-architecture.png
diff --git a/docs/jai-standalone-architecture.png b/docs/jai-standalone-architecture.png
diff --git a/docs/jai-worflow-anatomy.png b/docs/jai-worflow-anatomy.png
diff --git a/image/my-workflow.svg → jai-workflow-core/image/my-workflow.svg b/image/my-workflow.svg → jai-workflow-core/image/my-workflow.svg
diff --git a/jai-workflow-core/pom.xml b/jai-workflow-core/pom.xml
@@ -0,0 +1,71 @@
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
+    <modelVersion>4.0.0</modelVersion>
+    <parent>
+        <groupId>com.github.czelabueno</groupId>
+        <artifactId>jai-workflow-parent</artifactId>
+        <version>${revision}</version>
+    </parent>
+
+    <artifactId>jai-workflow-core</artifactId>
+    <name>JavAI Workflow :: Core</name>
+    <url>https://github.com/czelabueno/langchain4j-workflow</url>
+    <description>jAI Workflow: Build advanced agentic java AI applications based on workflows</description>
+
+    <dependencies>
+        <dependency>
+            <groupId>org.projectlombok</groupId>
+            <artifactId>lombok</artifactId>
+            <scope>provided</scope>
+        </dependency>
+
+        <dependency>
+            <groupId>guru.nidi</groupId>
+            <artifactId>graphviz-java</artifactId>
+        </dependency>
+
+        <dependency>
+            <groupId>org.graalvm.js</groupId>
+            <artifactId>js</artifactId>
+            <scope>runtime</scope>
+        </dependency>
+
+        <dependency>
+            <groupId>org.slf4j</groupId>
+            <artifactId>slf4j-api</artifactId>
+        </dependency>
+
+        <!-- Test libraries -->
+        <dependency>
+            <groupId>org.junit.jupiter</groupId>
+            <artifactId>junit-jupiter-engine</artifactId>
+            <version>${junit.version}</version>
+            <scope>test</scope>
+        </dependency>
+
+        <dependency>
+            <groupId>org.mockito</groupId>
+            <artifactId>mockito-junit-jupiter</artifactId>
+            <scope>test</scope>
+        </dependency>
+
+        <dependency>
+            <groupId>org.assertj</groupId>
+            <artifactId>assertj-core</artifactId>
+            <version>${assertj.version}</version>
+            <scope>test</scope>
+        </dependency>
+
+        <dependency>
+            <groupId>org.tinylog</groupId>
+            <artifactId>tinylog-impl</artifactId>
+            <scope>test</scope>
+        </dependency>
+
+        <dependency>
+            <groupId>org.tinylog</groupId>
+            <artifactId>slf4j-tinylog</artifactId>
+            <scope>test</scope>
+        </dependency>
+    </dependencies>
+</project>