DO NOT MERGE - Module refactoring

[markpollack]

This is an experiment to refactor the spring-ai-core module.

What was done.

Extract vector store and advisor functionality from spring-ai-core

• Created new module spring-ai-vector-store from spring-ai-core functionality.

  • Vector Stores can be used without an AI Models so it was moved outside spring-ai-core.

• Split advisor functionality into three new modules:
  * advisor-memory: Memory-based chat advisors
  * advisor-rag: Retrieval Augmentation Generation advisors
  * advisor-vector-store: Vector store based advisors

What wasn't done, but thought about.

Considered moving the Document package out into a new module or put in spring-ai-vector-store but recent research into the nature of advanced conversational management, make it seem like there are patterns - currently contained in the document oriented rag module - that apply to both message and document, so leaving it there.

Similarly considered moving the advanced RAG functionality to it's own module, but due to similaries with retrieving messages, left it in spring-ai-core for now.

Considered moving the ETL interfaces and classes into it's own package, but it seems too fine grained. Keeping the base interfaces in spring-ai-core with only JDK or 'core' based implementations (FileReader, JsonReader via Jackson) seems sufficient and any implementation of the ETL interfaces that bring in other dependencies are already separate modules.

[asaikali]

I think spring-ai has multiple levels of abstraction within it. At the lowest level is model access for chat, embedding, image ... etc. this is the lowest level API where portability across providers is the key, I think of this layer as the equivalent of JDBC in the SQL world.

The model access layer is an API/SPI with multiple SPI implementations. Ideally the implementer of the SPI maintain their own implementation so a key factor of the model access layer is backward comparability, and stability of interfaces.

The ChatClient, Advisors, ... etc. are really higher level interfaces where the goal is provide an ergonomic APIs for developers to write applications. This is more like JPA rather than JDBC.

Extracting a tiny model-access-core module that defines the API and SPI abstractions provides a stable foundation on top which higher level features of spring-ai can be built. A model-client module can provide developer experience abstractions such as advisors and chat client on top of model-access-core.

Low level model access APIs make it possible to provider alternative higher level interfaces as the industry discovers better patterns for building AI apps. Within the Java persistence ecosystem there are many competing approaches to offer a better higher level API from JPA, to JOOQ, to JdbcTemplate, to SqlClient ... etc. all of these are enabled by the lower level JDBC API.

Can the first first step on module refactoring of spring-ai be just the extraction of the tiny model-access-core module?