Skip to content

Commit

Permalink
feat(interactive): Introduce Interactive Java/Python SDK (#3602)
Browse files Browse the repository at this point in the history
Introduce the SDK of GraphScope Interactive. Including `Java SDK` and
`Python SDK`.
The most part of the code in SDKs are automatically generated from the
OpenAPI specification yaml(graphscope_interactive.yaml) using tools
provided by [OpenAPI](https://openapi-generator.tech). We integrate the
generated code and cypher/gremlin driver with some wrapper code.

- [x] The generated code `SHOULD not` be included in the codebase, user
can refer to `flex/interactive/sdk/README.md` to generate the SDKs if
interested.
- [x] Providing some examples about how to use the SDKs.
- [x] Add tests in Interactive CI
- [x] Also add docs for Interactive SDK.


For example usage of Python/Java SDK, see `BasicExample.java` and
`basic_example.py`.
  • Loading branch information
zhanglei1949 authored Apr 30, 2024
1 parent b4e7ea6 commit d3542e5
Show file tree
Hide file tree
Showing 46 changed files with 7,689 additions and 209 deletions.
29 changes: 29 additions & 0 deletions .github/workflows/hqps-db-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,35 @@ jobs:
bash hqps_admin_test.sh ${TMP_INTERACTIVE_WORKSPACE} ./engine_config_test.yaml ${GS_TEST_DIR}
sed -i 's/default_graph: modern_graph/default_graph: ldbc/g' ./engine_config_test.yaml
- name: Build and test Interactive Java/Python SDK
env:
FLEX_DATA_DIR: ${{ github.workspace }}/flex/interactive/examples/modern_graph
TMP_INTERACTIVE_WORKSPACE: /tmp/temp_workspace
LD_LIBRARY_PATH: /usr/local/lib
run: |
cd ${GITHUB_WORKSPACE}/flex/interactive/sdk/
bash generate_sdk.sh -g java
bash generate_sdk.sh -g python
cd java
mvn clean install -DskipTests
cd ../python
pip3 install -r test-requirements.txt
pip3 install .
rm -rf ${TMP_INTERACTIVE_WORKSPACE}
cd ${GITHUB_WORKSPACE}/flex/build/
SCHEMA_FILE=${GITHUB_WORKSPACE}/flex/interactive/examples/modern_graph/graph.yaml
BULK_LOAD_FILE=${GITHUB_WORKSPACE}/flex/interactive/examples/modern_graph/bulk_load.yaml
mkdir -p ${TMP_INTERACTIVE_WORKSPACE}/data/modern_graph/
cp ${SCHEMA_FILE} ${TMP_INTERACTIVE_WORKSPACE}/data/modern_graph/graph.yaml
GLOG_v=10 ./bin/bulk_loader -g ${SCHEMA_FILE} -l ${BULK_LOAD_FILE} -d ${TMP_INTERACTIVE_WORKSPACE}/data/modern_graph/indices/
cd ${GITHUB_WORKSPACE}/flex/tests/hqps
sed -i 's/default_graph: ldbc/default_graph: modern_graph/g' ./engine_config_test.yaml
bash hqps_sdk_test.sh ${TMP_INTERACTIVE_WORKSPACE} ./engine_config_test.yaml java
bash hqps_sdk_test.sh ${TMP_INTERACTIVE_WORKSPACE} ./engine_config_test.yaml python
sed -i 's/default_graph: modern_graph/default_graph: ldbc/g' ./engine_config_test.yaml
- name: Sample Query test
env:
GS_TEST_DIR: ${{ github.workspace }}/gstest
Expand Down
35 changes: 35 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,10 @@ docs/source/reference/
docs/reference/generated/
docs/reference/networkx/generated/
docs/_build/
docs/flex/interactive/development/java
!docs/flex/interactive/development/java/java_sdk.mod
docs/flex/interactive/development/python
!docs/flex/interactive/development/python/python_sdk.mod

analytical_engine/build/

Expand Down Expand Up @@ -100,4 +104,35 @@ flex/interactive/examples/sf0.1-raw/
flex/interactive/.running
flex/interactive/.env

# interactive sdk related
flex/interactive/sdk/java/.github/
flex/interactive/sdk/java/.gitignore
flex/interactive/sdk/java/docs/
flex/interactive/sdk/java/build.sbt
flex/interactive/sdk/java/build.gradle
flex/interactive/sdk/java/gradle.properties
flex/interactive/sdk/java/gradlew.bat
flex/interactive/sdk/java/gradlew
flex/interactive/sdk/java/.travis.yml
flex/interactive/sdk/java/git_push.sh
flex/interactive/sdk/java/gradle/
flex/interactive/sdk/java/.openapi-generator/*
flex/interactive/sdk/java/src/main/AndroidManifest.xml
flex/interactive/sdk/java/.openapi-generate/
flex/interactive/sdk/java/settings.gradle
flex/interactive/sdk/java/src/main/java/com/alibaba/graphscope/interactive/openapi/

flex/interactive/sdk/python/.github/
flex/interactive/sdk/python/.gitignore
flex/interactive/sdk/python/docs/
flex/interactive/sdk/python/git_push.sh
flex/interactive/sdk/python/.gitlab-ci.yml
flex/interactive/sdk/python/.openapi-generator/
flex/interactive/sdk/python/dist/
flex/interactive/sdk/python/.travis.yml
flex/interactive/sdk/python/tox.ini
flex/interactive/sdk/python/pyproject.toml
flex/interactive/sdk/python/interactive_sdk/openapi/


**/.cache/
12 changes: 11 additions & 1 deletion docs/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,16 @@ clean:
@mkdir -p "$(BUILDDIR)/${TAG}/$@/reference/"
@cat Doxyfile-flex | sed 's/_build\/latest\/html\/reference\//$(BUILDDIR)\/${TAG}\/$@\/reference\//g' | doxygen -

doxygen:
interactive-sdk:
@echo "Building interactive sdk doc to $(BUILDDIR)/${TAG}/interactive-sdk"
@cd ${WORKING_DIR}/../flex/interactive/sdk/ && ./generate_sdk.sh -g java \
&& ./generate_sdk.sh -g python

# copy the generated files to ${WORKING_DIR}/flex/interactive/development/
@cp -r ${WORKING_DIR}/../flex/interactive/sdk/java/docs/* ${WORKING_DIR}/flex/interactive/development/java/
@cp -r ${WORKING_DIR}/../flex/interactive/sdk/python/docs/* ${WORKING_DIR}/flex/interactive/development/python/


doxygen: interactive-sdk
@mkdir -p _build
@doxygen Doxyfile
4 changes: 3 additions & 1 deletion docs/flex/interactive/dev_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,7 @@ maxdepth: 2
---
development/cypher_procedure
development/cpp_procedure
development/admin_service
development/restful_api
development/java/java_sdk
development/python/python_sdk
```
202 changes: 202 additions & 0 deletions docs/flex/interactive/development/java/java_sdk.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,202 @@
# Java SDK Reference

The Interactive Java SDK Reference is a comprehensive guide for developers looking to integrate the Interactive service into their Java applications. This SDK allows users to seamlessly connect to Interactive and leverage its powerful features for graph management, stored procedure management, and query execution.


## Requirements

Building the API client library requires:
1. Java 1.8+
2. Maven (3.8.3+)/Gradle (7.2+)

## Installation

To install the API client library to your local Maven repository, simply execute:

```shell
git clone https://github.com/alibaba/GraphScope.git
cd GraphScope/flex/interactive/sdk/java
mvn clean install
```

To deploy it to a remote Maven repository instead, configure the settings of the repository and execute:

```shell
mvn clean deploy
```

Refer to the [OSSRH Guide](http://central.sonatype.org/pages/ossrh-guide.html) for more information.

### Maven users

Add this dependency to your project's POM:

```xml
<dependency>
<groupId>com.alibaba.graphscope</groupId>
<artifactId>interactive-java-sdk</artifactId>
<version>0.0.3</version>
<scope>compile</scope>
</dependency>
```

### Others

At first generate the JAR by executing:

```shell
mvn clean package
```

Then manually install the following JARs:

* `target/interactive-java-sdk-0.0.3.jar`
* `target/lib/*.jar`

## Getting Started

First, install and start the interactive service via [Interactive Getting Started](https://graphscope.io/docs/flex/interactive/getting_started), and you will get the endpoint for the Interactive service.

```bash
Interactive Service is listening at ${INTERACTIVE_ENDPOINT}.
```

Then, connect to the interactive endpoint, and try to run a simple query with following code.

```java
package com.alibaba.graphscope;

import com.alibaba.graphscope.interactive.client.Driver;
import com.alibaba.graphscope.interactive.client.Session;

public class GettingStarted {
public static void main(String[] args) {
//get endpoint from command line
if (args.length != 1) {
System.out.println("Usage: <endpoint>");
return;
}
String endpoint = args[0];
Driver driver = Driver.connect(endpoint);
Session session = driver.session();

// start a query
// run cypher query
try (org.neo4j.driver.Session neo4jSession = driver.getNeo4jSession()) {
org.neo4j.driver.Result result = neo4jSession.run("MATCH(a) return COUNT(a);");
System.out.println("result: " + result.toString());
}
return;
}
}
```

For more a more detailed example, please refer to [Java SDK Example](https://github.com/alibaba/GraphScope/flex/interactive/sdk/examples/java/interactive-sdk-example/)

## Documentation for Service APIs

All URIs are relative to `${INTERACTIVE_ENDPOINT}`

Class | Method | HTTP request | Description
------------ | ------------- | ------------- | -------------
*AdminServiceGraphManagementApi* | [**createDataloadingJob**](./AdminServiceGraphManagementApi.md#createDataloadingJob) | **POST** /v1/graph/{graph_id}/dataloading |
*AdminServiceGraphManagementApi* | [**createGraph**](./AdminServiceGraphManagementApi.md#createGraph) | **POST** /v1/graph |
*AdminServiceGraphManagementApi* | [**deleteGraph**](./AdminServiceGraphManagementApi.md#deleteGraph) | **DELETE** /v1/graph/{graph_id} |
*AdminServiceGraphManagementApi* | [**getGraph**](./AdminServiceGraphManagementApi.md#getGraph) | **GET** /v1/graph/{graph_id} |
*AdminServiceGraphManagementApi* | [**getSchema**](./AdminServiceGraphManagementApi.md#getSchema) | **GET** /v1/graph/{graph_id}/schema |
*AdminServiceGraphManagementApi* | [**listGraphs**](./AdminServiceGraphManagementApi.md#listGraphs) | **GET** /v1/graph |
*AdminServiceJobManagementApi* | [**deleteJobById**](./AdminServiceJobManagementApi.md#deleteJobById) | **DELETE** /v1/job/{job_id} |
*AdminServiceJobManagementApi* | [**getJobById**](./AdminServiceJobManagementApi.md#getJobById) | **GET** /v1/job/{job_id} |
*AdminServiceJobManagementApi* | [**listJobs**](./AdminServiceJobManagementApi.md#listJobs) | **GET** /v1/job |
*AdminServiceProcedureManagementApi* | [**createProcedure**](./AdminServiceProcedureManagementApi.md#createProcedure) | **POST** /v1/graph/{graph_id}/procedure |
*AdminServiceProcedureManagementApi* | [**deleteProcedure**](./AdminServiceProcedureManagementApi.md#deleteProcedure) | **DELETE** /v1/graph/{graph_id}/procedure/{procedure_id} |
*AdminServiceProcedureManagementApi* | [**getProcedure**](./AdminServiceProcedureManagementApi.md#getProcedure) | **GET** /v1/graph/{graph_id}/procedure/{procedure_id} |
*AdminServiceProcedureManagementApi* | [**listProcedures**](./AdminServiceProcedureManagementApi.md#listProcedures) | **GET** /v1/graph/{graph_id}/procedure |
*AdminServiceProcedureManagementApi* | [**updateProcedure**](./AdminServiceProcedureManagementApi.md#updateProcedure) | **PUT** /v1/graph/{graph_id}/procedure/{procedure_id} |
*AdminServiceServiceManagementApi* | [**getServiceStatus**](./AdminServiceServiceManagementApi.md#getServiceStatus) | **GET** /v1/service/status |
*AdminServiceServiceManagementApi* | [**restartService**](./AdminServiceServiceManagementApi.md#restartService) | **POST** /v1/service/restart |
*AdminServiceServiceManagementApi* | [**startService**](./AdminServiceServiceManagementApi.md#startService) | **POST** /v1/service/start |
*AdminServiceServiceManagementApi* | [**stopService**](./AdminServiceServiceManagementApi.md#stopService) | **POST** /v1/service/stop |
*GraphServiceEdgeManagementApi* | [**addEdge**](./GraphServiceEdgeManagementApi.md#addEdge) | **POST** /v1/graph/{graph_id}/edge | Add edge to the graph
*GraphServiceEdgeManagementApi* | [**deleteEdge**](./GraphServiceEdgeManagementApi.md#deleteEdge) | **DELETE** /v1/graph/{graph_id}/edge | Remove edge from the graph
*GraphServiceEdgeManagementApi* | [**getEdge**](./GraphServiceEdgeManagementApi.md#getEdge) | **GET** /v1/graph/{graph_id}/edge | Get the edge&#39;s properties with src and dst vertex primary keys.
*GraphServiceEdgeManagementApi* | [**updateEdge**](./GraphServiceEdgeManagementApi.md#updateEdge) | **PUT** /v1/graph/{graph_id}/edge | Update edge&#39;s property
*GraphServiceVertexManagementApi* | [**addVertex**](./GraphServiceVertexManagementApi.md#addVertex) | **POST** /v1/graph/{graph_id}/vertex | Add vertex to the graph
*GraphServiceVertexManagementApi* | [**deleteVertex**](./GraphServiceVertexManagementApi.md#deleteVertex) | **DELETE** /v1/graph/{graph_id}/vertex | Remove vertex from the graph
*GraphServiceVertexManagementApi* | [**getVertex**](./GraphServiceVertexManagementApi.md#getVertex) | **GET** /v1/graph/{graph_id}/vertex | Get the vertex&#39;s properties with vertex primary key.
*GraphServiceVertexManagementApi* | [**updateVertex**](./GraphServiceVertexManagementApi.md#updateVertex) | **PUT** /v1/graph/{graph_id}/vertex | Update vertex&#39;s property
*QueryServiceApi* | [**procCall**](./QueryServiceApi.md#procCall) | **POST** /v1/graph/{graph_id}/query | run queries on graph


## Documentation for Data Structures

- [BaseEdgeType](./BaseEdgeType.md)
- [BaseEdgeTypeVertexTypePairRelationsInner](./BaseEdgeTypeVertexTypePairRelationsInner.md)
- [BaseEdgeTypeVertexTypePairRelationsInnerXCsrParams](./BaseEdgeTypeVertexTypePairRelationsInnerXCsrParams.md)
- [BasePropertyMeta](./BasePropertyMeta.md)
- [BaseVertexType](./BaseVertexType.md)
- [BaseVertexTypeXCsrParams](./BaseVertexTypeXCsrParams.md)
- [Collection](./Collection.md)
- [CollectiveResults](./CollectiveResults.md)
- [Column](./Column.md)
- [ColumnMapping](./ColumnMapping.md)
- [CreateEdgeType](./CreateEdgeType.md)
- [CreateGraphRequest](./CreateGraphRequest.md)
- [CreateGraphResponse](./CreateGraphResponse.md)
- [CreateGraphSchemaRequest](./CreateGraphSchemaRequest.md)
- [CreateProcedureRequest](./CreateProcedureRequest.md)
- [CreateProcedureResponse](./CreateProcedureResponse.md)
- [CreatePropertyMeta](./CreatePropertyMeta.md)
- [CreateVertexType](./CreateVertexType.md)
- [EdgeData](./EdgeData.md)
- [EdgeMapping](./EdgeMapping.md)
- [EdgeMappingDestinationVertexMappingsInner](./EdgeMappingDestinationVertexMappingsInner.md)
- [EdgeMappingSourceVertexMappingsInner](./EdgeMappingSourceVertexMappingsInner.md)
- [EdgeMappingSourceVertexMappingsInnerColumn](./EdgeMappingSourceVertexMappingsInnerColumn.md)
- [EdgeMappingTypeTriplet](./EdgeMappingTypeTriplet.md)
- [EdgeRequest](./EdgeRequest.md)
- [Element](./Element.md)
- [FixedChar](./FixedChar.md)
- [FixedCharChar](./FixedCharChar.md)
- [GSDataType](./GSDataType.md)
- [GetEdgeType](./GetEdgeType.md)
- [GetGraphResponse](./GetGraphResponse.md)
- [GetGraphSchemaResponse](./GetGraphSchemaResponse.md)
- [GetProcedureResponse](./GetProcedureResponse.md)
- [GetPropertyMeta](./GetPropertyMeta.md)
- [GetVertexType](./GetVertexType.md)
- [JobResponse](./JobResponse.md)
- [JobStatus](./JobStatus.md)
- [KeyValue](./KeyValue.md)
- [LongText](./LongText.md)
- [Parameter](./Parameter.md)
- [PrimitiveType](./PrimitiveType.md)
- [Property](./Property.md)
- [PropertyArray](./PropertyArray.md)
- [QueryRequest](./QueryRequest.md)
- [Record](./Record.md)
- [SchemaMapping](./SchemaMapping.md)
- [SchemaMappingLoadingConfig](./SchemaMappingLoadingConfig.md)
- [SchemaMappingLoadingConfigFormat](./SchemaMappingLoadingConfigFormat.md)
- [ServiceStatus](./ServiceStatus.md)
- [StartServiceRequest](./StartServiceRequest.md)
- [StoredProcedureMeta](./StoredProcedureMeta.md)
- [StringType](./StringType.md)
- [StringTypeString](./StringTypeString.md)
- [TemporalType](./TemporalType.md)
- [TimeStampType](./TimeStampType.md)
- [TypedValue](./TypedValue.md)
- [UpdateProcedureRequest](./UpdateProcedureRequest.md)
- [VarChar](./VarChar.md)
- [VarCharVarChar](./VarCharVarChar.md)
- [VertexData](./VertexData.md)
- [VertexMapping](./VertexMapping.md)
- [VertexRequest](./VertexRequest.md)


<a id="documentation-for-authorization"></a>
## Documentation for Authorization

Authentication is not supported yet, and we will be introducing authorization-related implementation in the near future.


Loading

0 comments on commit d3542e5

Please sign in to comment.