From d76675b037f9268c24f08c3b5c20f4aac1a84187 Mon Sep 17 00:00:00 2001 From: katarinasupe Date: Tue, 25 Apr 2023 15:16:31 +0200 Subject: [PATCH 1/5] Add draft import overview --- gqlalchemy/getting-started.md | 6 +++++- gqlalchemy/import-data.md | 33 +++++++++++++++++++++++++++++++++ sidebars/sidebarsGQLAlchemy.js | 1 + 3 files changed, 39 insertions(+), 1 deletion(-) create mode 100644 gqlalchemy/import-data.md diff --git a/gqlalchemy/getting-started.md b/gqlalchemy/getting-started.md index 0b2702398a7..8e386eb5671 100644 --- a/gqlalchemy/getting-started.md +++ b/gqlalchemy/getting-started.md @@ -22,7 +22,11 @@ source](/installation.md#source). If you are using [Conda](https://docs.conda.io GQLAlchemy can't be installed with Python 3.11 [(#203)](https://github.com/memgraph/gqlalchemy/issues/203) and on Windows with Python > 3.9 [(#179)](https://github.com/memgraph/gqlalchemy/issues/179). If this is currently a blocker for you, please let us know by commenting on opened issues. ::: -### 2. Learn how to use GQLAlchemy +### 2. Connect to Memgraph + +Check the [Python quick start guide](/memgraph/connect-to-memgraph/drivers/python) on Memgraph documentation to learn how to connect to Memgraph using GQLAlchemy. + +### 3. Learn how to use GQLAlchemy With the help of the [How-to guides](/how-to-guides/overview.md) you can learn how to use GQLAlchemy's features, such as object graph mapper and query builder. diff --git a/gqlalchemy/import-data.md b/gqlalchemy/import-data.md new file mode 100644 index 00000000000..c7191e27020 --- /dev/null +++ b/gqlalchemy/import-data.md @@ -0,0 +1,33 @@ +--- +id: import-data +title: Import data +sidebar_label: Import data +slug: /import-data +--- + +You can import data in the following formats: + +- CSV + - call Cypher query -> Memgraph docs + - transform CSV into graph -> GQLA how-to + +- JSON + - call Cypher query -> Memgraph docs + +- Parquet, ORC or IPC/Feather/Arrow + - transform table data from file into graph -> GQLA how-to + +- NetworkX/PyG/DGL graph + - transform into Memgraph graph + +- Kafka/RedPanda/Pulsar data stream + - Run Cypher to connect to data stream -> GQLA how-to + + + +If you want to create the data directly from code use the Object Graph Mapper or query builder. (links to how-to guides) + + +:::tip +Fastest way of data import +::: diff --git a/sidebars/sidebarsGQLAlchemy.js b/sidebars/sidebarsGQLAlchemy.js index c77cd2d6823..ad9702d89eb 100644 --- a/sidebars/sidebarsGQLAlchemy.js +++ b/sidebars/sidebarsGQLAlchemy.js @@ -2,6 +2,7 @@ module.exports = { gqlalchemy: [ "getting-started", "installation", + "import-data", { type: "category", label: "How-to guides", From 023e0bc7b716a5e02fd1335359924b9ab97096da Mon Sep 17 00:00:00 2001 From: katarinasupe Date: Thu, 27 Apr 2023 16:25:58 +0200 Subject: [PATCH 2/5] Update import overview --- docusaurus.config.js | 4 ++ gqlalchemy/how-to-guides/ogm.md | 15 +++++++ gqlalchemy/how-to-guides/query-builder.md | 39 +++++++++++++++++- gqlalchemy/import-data.md | 48 ++++++++++++++++------- 4 files changed, 91 insertions(+), 15 deletions(-) diff --git a/docusaurus.config.js b/docusaurus.config.js index 5137b36cd53..1cc212c93e2 100644 --- a/docusaurus.config.js +++ b/docusaurus.config.js @@ -214,6 +214,10 @@ module.exports = { label: "Installation", to: "/gqlalchemy/installation", }, + { + label: "Import data", + to: "/gqlalchemy/import-data", + }, { label: "How-to guides", to: "/gqlalchemy/how-to-guides", diff --git a/gqlalchemy/how-to-guides/ogm.md b/gqlalchemy/how-to-guides/ogm.md index d44d479f1d1..326e4d10ddb 100644 --- a/gqlalchemy/how-to-guides/ogm.md +++ b/gqlalchemy/how-to-guides/ogm.md @@ -341,6 +341,21 @@ The indexes will be set on class definition, before instantiation. This ensures print(db.get_indexes()) ``` +The other way to create indexes is by creating an instance of `MemgraphIndex` class. For example, to create label index `NodeOne` and label-property index `NodeOne(name)`, run the following code: + +```python +from gqlalchemy import Memgraph +from gqlalchemy.models import MemgraphIndex + +db = Memgraph() + +index1 = MemgraphIndex("NodeOne") +index2 = MemgraphIndex("NodeOne", "name") + +db.create_index(index1) +db.create_index(index2) +``` + To learn more about indexes, head over to the [indexing reference guide](/memgraph/reference-guide/indexing). ## Create constraints diff --git a/gqlalchemy/how-to-guides/query-builder.md b/gqlalchemy/how-to-guides/query-builder.md index d0a814ad73d..a166cb446ca 100644 --- a/gqlalchemy/how-to-guides/query-builder.md +++ b/gqlalchemy/how-to-guides/query-builder.md @@ -35,6 +35,7 @@ Through this guide, you will learn how to use GQLAlchemy query builder to: - [**Call procedures**](#call-procedures) - [**Call procedure with no arguments**](#call-procedure-with-no-arguments) - [**Call procedure with arguments**](#call-procedure-with-arguments) +- [**Load CSV file**](#load-csv-file) >Hopefully, this guide will teach you how to properly use GQLAlchemy query builder. If you >have any more questions, join our community and ping us on [Discord](https://discord.gg/memgraph). @@ -1241,7 +1242,8 @@ RETURN objects; -## Full code example +
+ Code example using all of the above mentioned queries ```python from gqlalchemy import create, merge, Memgraph, match, models, call @@ -1490,7 +1492,42 @@ results = list( print("Load from URL with argument:", results, "\n") ``` +
+## Load CSV file + +To load a CSV file using query builder, use the `load_csv()` procedure. Here is an example CSV file: +``` +id,name,age,city +100,Daniel,30,London +101,Alex,15,Paris +102,Sarah,17,London +103,Mia,25,Zagreb +104,Lucy,21,Paris +``` + +To load it, run the following code: + +```python +from gqlalchemy import load_csv, Memgraph +from gqlalchemy.utilities import CypherVariable + +db = Memgraph() + +load_csv( + path="/path-to/people_nodes.csv", header=True, row="row" + ) + .create() + .node( + variable="n", + labels="Person", + id=CypherVariable(name="row.id"), + name=CypherVariable(name="row.name"), + age=CypherVariable(name="ToInteger(row.age)"), + city=CypherVariable(name="row.city"), + ) + .execute() +``` >Hopefully, this guide has taught you how to properly use GQLAlchemy query builder. If you >have any more questions, join our community and ping us on [Discord](https://discord.gg/memgraph). diff --git a/gqlalchemy/import-data.md b/gqlalchemy/import-data.md index c7191e27020..b8c480233e3 100644 --- a/gqlalchemy/import-data.md +++ b/gqlalchemy/import-data.md @@ -6,28 +6,48 @@ slug: /import-data --- You can import data in the following formats: +- [**CSV**](#csv) +- [**JSON**](#json) +- [**Parquet, ORC or IPC/Feather/Arrow**](#parquet-orc-or-ipcfeatherarrow) +- [**Python graphs - NetworkX, PyG or DGL graph**](#python-graphs---networkx-pyg-or-dgl-graph) +- [**Kafka, RedPanda or Pulsar data stream**](#kafka-redpanda-or-pulsar-data-stream) -- CSV - - call Cypher query -> Memgraph docs - - transform CSV into graph -> GQLA how-to +Besides that, you can create the data directly from code using the [**object graph mapper**](/gqlalchemy/how-to-guides/ogm) or [**query builder**](/gqlalchemy/how-to-guides/query-builder). -- JSON - - call Cypher query -> Memgraph docs -- Parquet, ORC or IPC/Feather/Arrow - - transform table data from file into graph -> GQLA how-to +:::tip +Fastest way to import data into Memgraph is using the [LOAD CSV clause](/memgraph/import-data/load-csv-clause). Don't forget to [set up indexes](/memgraph/next/how-to-guides/indexes) first with the `CREATE INDEX` clause! You can set them up by [executing the Cypher query](/memgraph/connect-to-memgraph/drivers/python) or with [object graph mapper](/gqlalchemy/how-to-guides/ogm#create-indexes). +::: -- NetworkX/PyG/DGL graph - - transform into Memgraph graph +## CSV -- Kafka/RedPanda/Pulsar data stream - - Run Cypher to connect to data stream -> GQLA how-to +To import CSV file into Memgraph via GQLAlchemy, you can use the [`LOAD CSV` clause](/memgraph/import-data/load-csv-clause). That clause can be used by [executing the Cypher query](/memgraph/connect-to-memgraph/drivers/python) or by [building the query with the query builder](/gqlalchemy/how-to-guides/query-builder#load-csv-file). Another way of importing CSV data into Memgraph is by [translating it into a graph](/gqlalchemy/how-to-guides/table-to-graph-importer). +## JSON +To import JSON file into Memgraph via GQLAlchemy, you can call procedures from the [`json_util` module](/mage/query-modules/python/json-util) from MAGE. If the JSON data is in particular format, you can call the [`import_util.json()` procedure](/mage/query-modules/python/import-util#jsonpath) from MAGE. The procedures can be called by [executing Cypher queries](/memgraph/connect-to-memgraph/drivers/python) or [using the query builder](/gqlalchemy/how-to-guides/query-builder#call-procedures). -If you want to create the data directly from code use the Object Graph Mapper or query builder. (links to how-to guides) +## Parquet, ORC or IPC/Feather/Arrow -:::tip -Fastest way of data import +To import Parquet, ORC or IPC/Feather/Arrow file into Memgraph via GQLAlchemy, [transform table data from file into a graph](/gqlalchemy/how-to-guides/table-to-graph-importer). + +:::note +If you want to read from a file system not currently supported by GQLAlchemy, or use a file type currently not readable, you can implement your own by [making a custom file system importer](/gqlalchemy/how-to-guides/custom-file-system-importer). ::: + + +## Python graphs - NetworkX, PyG or DGL graph + +To import NetworkX, PyG or DGL graph into Memgraph via GQLAlchemy, [transform the source graph into Memgraph graph](/gqlalchemy/how-to-guides/networkx). + +## Kafka, RedPanda or Pulsar data stream + +To consume Kafka, RedPanda or Pulsar data stream, you can write [appropriate Cypher queries](/memgraph/import-data/data-streams/manage-streams) and [execute](/memgraph/connect-to-memgraph/drivers/python) them, or use GQLAlchemy stream manager for [Kafka/RedPanda](/gqlalchemy/how-to-guides/streams/manage-kafka-streams) or [Pulsar](/gqlalchemy/how-to-guides/streams/manage-pulsar-streams) streams. + + +## Learn more + +To learn how to utilize the GQLAlchemy library with Memgraph, check out the [how-to guides](/gqlalchemy/how-to-guides) or sign up for the [Getting started with Memgraph and Python course](https://app.livestorm.co/memgraph/getting-started-with-memgraph-and-python-on-demand). + + From b93a43c09e51b044072d052b7f8c3d40cbc739ca Mon Sep 17 00:00:00 2001 From: katarinasupe Date: Thu, 27 Apr 2023 16:29:19 +0200 Subject: [PATCH 3/5] Update how-to overview --- gqlalchemy/how-to-guides/overview.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/gqlalchemy/how-to-guides/overview.md b/gqlalchemy/how-to-guides/overview.md index d9596dbad11..d8384bcdf73 100644 --- a/gqlalchemy/how-to-guides/overview.md +++ b/gqlalchemy/how-to-guides/overview.md @@ -71,3 +71,20 @@ any of the graph algorithms. Learn how to use on-disk storage in the following guide: - [**On-disk storage**](/how-to-guides/on-disk-storage/on-disk-storage.md) + +## Graph projections + +As subgraphs are mainly used with Memgraph's query modules (graph algorithms), +QueryBuilder's `call()` method enables specifying the subgraph to use with a +certain algorithm. + +- [**Create a graph projection**](/gqlalchemy/how-to-guides/query-builder/graph-projection) + +## Transform Python graphs into Memgraph graphs + +GQLAlchemy holds transformations that can transform NetworkX, PyG and DGL graphs +into Memgraph graphs. These transformations take the source graph object and +translate it to the appropriate Cypher queries. The Cypher queries are then +executed to create a graph inside Memgraph. + +- [**Import NetworkX graph into Memgraph**](/gqlalchemy/how-to-guides/networkx) From cd2cec5d349882c8fe2cced89abd4bfa9e6cac24 Mon Sep 17 00:00:00 2001 From: katarinasupe Date: Fri, 28 Apr 2023 15:34:08 +0200 Subject: [PATCH 4/5] Vlasta's comments --- gqlalchemy/getting-started.md | 2 +- gqlalchemy/import-data.md | 8 ++++---- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/gqlalchemy/getting-started.md b/gqlalchemy/getting-started.md index 8e386eb5671..838a51b539d 100644 --- a/gqlalchemy/getting-started.md +++ b/gqlalchemy/getting-started.md @@ -24,7 +24,7 @@ GQLAlchemy can't be installed with Python 3.11 [(#203)](https://github.com/memgr ### 2. Connect to Memgraph -Check the [Python quick start guide](/memgraph/connect-to-memgraph/drivers/python) on Memgraph documentation to learn how to connect to Memgraph using GQLAlchemy. +Check the [Python quick start guide](/memgraph/connect-to-memgraph/drivers/python) to learn how to connect to Memgraph using GQLAlchemy. ### 3. Learn how to use GQLAlchemy diff --git a/gqlalchemy/import-data.md b/gqlalchemy/import-data.md index b8c480233e3..e92e02dab0a 100644 --- a/gqlalchemy/import-data.md +++ b/gqlalchemy/import-data.md @@ -12,11 +12,11 @@ You can import data in the following formats: - [**Python graphs - NetworkX, PyG or DGL graph**](#python-graphs---networkx-pyg-or-dgl-graph) - [**Kafka, RedPanda or Pulsar data stream**](#kafka-redpanda-or-pulsar-data-stream) -Besides that, you can create the data directly from code using the [**object graph mapper**](/gqlalchemy/how-to-guides/ogm) or [**query builder**](/gqlalchemy/how-to-guides/query-builder). +Besides that, you can create data directly from code using the [**object graph mapper**](/gqlalchemy/how-to-guides/ogm) or [**query builder**](/gqlalchemy/how-to-guides/query-builder). :::tip -Fastest way to import data into Memgraph is using the [LOAD CSV clause](/memgraph/import-data/load-csv-clause). Don't forget to [set up indexes](/memgraph/next/how-to-guides/indexes) first with the `CREATE INDEX` clause! You can set them up by [executing the Cypher query](/memgraph/connect-to-memgraph/drivers/python) or with [object graph mapper](/gqlalchemy/how-to-guides/ogm#create-indexes). +The fastest way to import data into Memgraph is by using the [LOAD CSV clause](/memgraph/import-data/load-csv-clause). It's recommended to first [create indexes](/memgraph/next/how-to-guides/indexes) using the `CREATE INDEX` clause. You can create them by [executing the Cypher query](/memgraph/connect-to-memgraph/drivers/python) or using [object graph mapper](/gqlalchemy/how-to-guides/ogm#create-indexes). ::: ## CSV @@ -25,7 +25,7 @@ To import CSV file into Memgraph via GQLAlchemy, you can use the [`LOAD CSV` cla ## JSON -To import JSON file into Memgraph via GQLAlchemy, you can call procedures from the [`json_util` module](/mage/query-modules/python/json-util) from MAGE. If the JSON data is in particular format, you can call the [`import_util.json()` procedure](/mage/query-modules/python/import-util#jsonpath) from MAGE. The procedures can be called by [executing Cypher queries](/memgraph/connect-to-memgraph/drivers/python) or [using the query builder](/gqlalchemy/how-to-guides/query-builder#call-procedures). +To import JSON files into Memgraph via GQLAlchemy, you can call procedures from the [`json_util` module](/mage/query-modules/python/json-util) available in MAGE library. If the JSON data is formatted in a particular style, you can call the [`import_util.json()` procedure](/mage/query-modules/python/import-util#jsonpath) from MAGE. The procedures can be called by [executing Cypher queries](/memgraph/connect-to-memgraph/drivers/python) or [using the query builder](/gqlalchemy/how-to-guides/query-builder#call-procedures). ## Parquet, ORC or IPC/Feather/Arrow @@ -43,7 +43,7 @@ To import NetworkX, PyG or DGL graph into Memgraph via GQLAlchemy, [transform th ## Kafka, RedPanda or Pulsar data stream -To consume Kafka, RedPanda or Pulsar data stream, you can write [appropriate Cypher queries](/memgraph/import-data/data-streams/manage-streams) and [execute](/memgraph/connect-to-memgraph/drivers/python) them, or use GQLAlchemy stream manager for [Kafka/RedPanda](/gqlalchemy/how-to-guides/streams/manage-kafka-streams) or [Pulsar](/gqlalchemy/how-to-guides/streams/manage-pulsar-streams) streams. +To consume Kafka, RedPanda or Pulsar data stream, you can write a[appropriate Cypher queries](/memgraph/import-data/data-streams/manage-streams) and [execute](/memgraph/connect-to-memgraph/drivers/python) them, or use GQLAlchemy stream manager for [Kafka, RedPanda](/gqlalchemy/how-to-guides/streams/manage-kafka-streams) or [Pulsar](/gqlalchemy/how-to-guides/streams/manage-pulsar-streams) streams. ## Learn more From e792ab42ad3e905002c694c854d5db5414c902f3 Mon Sep 17 00:00:00 2001 From: katarinasupe Date: Fri, 28 Apr 2023 15:35:04 +0200 Subject: [PATCH 5/5] Add missing article --- gqlalchemy/import-data.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gqlalchemy/import-data.md b/gqlalchemy/import-data.md index e92e02dab0a..e1f465baeca 100644 --- a/gqlalchemy/import-data.md +++ b/gqlalchemy/import-data.md @@ -30,7 +30,7 @@ To import JSON files into Memgraph via GQLAlchemy, you can call procedures from ## Parquet, ORC or IPC/Feather/Arrow -To import Parquet, ORC or IPC/Feather/Arrow file into Memgraph via GQLAlchemy, [transform table data from file into a graph](/gqlalchemy/how-to-guides/table-to-graph-importer). +To import Parquet, ORC or IPC/Feather/Arrow file into Memgraph via GQLAlchemy, [transform table data from a file into a graph](/gqlalchemy/how-to-guides/table-to-graph-importer). :::note If you want to read from a file system not currently supported by GQLAlchemy, or use a file type currently not readable, you can implement your own by [making a custom file system importer](/gqlalchemy/how-to-guides/custom-file-system-importer).