Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added get_replica_connections function #72

Open
wants to merge 39 commits into
base: main
Choose a base branch
from
Open
Changes from 2 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
e7beff6
added get_replica_connections function
MuhammadQadora Oct 28, 2024
6a717ab
only return connections
MuhammadQadora Nov 6, 2024
0791c7e
removed flag
MuhammadQadora Nov 14, 2024
c77580e
updated to suite Coderabbit comments
MuhammadQadora Nov 14, 2024
5ee68b7
fixed function
MuhammadQadora Nov 14, 2024
c4dc33a
removed unnecessary variable
MuhammadQadora Nov 14, 2024
2c20a1e
Update falkordb.py
swilly22 Nov 14, 2024
99b6671
added error handling
MuhammadQadora Nov 14, 2024
dd8a4ad
removed error handler
MuhammadQadora Nov 14, 2024
8eed87f
added test
MuhammadQadora Nov 18, 2024
e646320
added tests
MuhammadQadora Nov 20, 2024
4a3d585
fixed path
MuhammadQadora Nov 20, 2024
a1262e0
updated
MuhammadQadora Nov 20, 2024
199a1e5
updated
MuhammadQadora Nov 20, 2024
71622fe
updated test
MuhammadQadora Nov 21, 2024
9f261fc
test for clusterCon and SentinelCon
MuhammadQadora Nov 21, 2024
9d21e95
removed wrong import
MuhammadQadora Nov 21, 2024
6aad7bd
fixed test
MuhammadQadora Nov 21, 2024
2a96be0
updated test
MuhammadQadora Nov 22, 2024
b693987
changed
MuhammadQadora Nov 24, 2024
519b9dd
updated
MuhammadQadora Nov 24, 2024
557f04f
added compose files
MuhammadQadora Dec 2, 2024
222cfb4
created deployments standalone, redis cluster and redis replication f…
MuhammadQadora Dec 16, 2024
a9f597e
testing
MuhammadQadora Dec 18, 2024
ac8471e
created deployments standalone, redis cluster and redis replication f…
MuhammadQadora Dec 19, 2024
cfd0864
created deployments standalone, redis cluster and redis replication f…
MuhammadQadora Dec 19, 2024
65e41d0
full path
MuhammadQadora Dec 19, 2024
d112322
removed exceptions
MuhammadQadora Dec 19, 2024
ad7cedd
moved docker files to docker folder
swilly22 Dec 22, 2024
ac4cfa4
changed image to use falkordb/falkordb:edge and overriden the entrypoint
MuhammadQadora Dec 26, 2024
5a444db
fixed entrypoint
MuhammadQadora Dec 26, 2024
4cc0b6a
added yml suffix
MuhammadQadora Dec 26, 2024
51d4d90
updated image tag
MuhammadQadora Dec 29, 2024
cb8cd2e
[WIP] improved over all test latency
swilly22 Dec 29, 2024
1ad3162
Merge branch 'main' of https://github.com/FalkorDB/falkordb-py into 7…
MuhammadQadora Dec 30, 2024
4d4c1ac
change compose files and removed the need for docker compose down
MuhammadQadora Jan 1, 2025
263e8bf
change compose files and removed the need for docker compose down
MuhammadQadora Jan 2, 2025
af4f748
removed echos
MuhammadQadora Jan 2, 2025
b1d6c41
enhanced waiting
MuhammadQadora Jan 2, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 21 additions & 2 deletions falkordb/falkordb.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
from .sentinel import *
from .graph import Graph
from typing import List, Union

import socket
# config command
LIST_CMD = "GRAPH.LIST"
CONFIG_CMD = "GRAPH.CONFIG"
Expand Down Expand Up @@ -75,6 +75,7 @@
dynamic_startup_nodes=True,
url=None,
address_remap=None,
decode_responses=True
):

conn = redis.Redis(
Expand All @@ -92,7 +93,7 @@
encoding_errors=encoding_errors,
charset=charset,
errors=errors,
decode_responses=True,
decode_responses=decode_responses,
retry_on_timeout=retry_on_timeout,
retry_on_error=retry_on_error,
ssl=ssl,
Expand Down Expand Up @@ -125,6 +126,7 @@
self.sentinel, self.service_name = Sentinel_Conn(conn, ssl)
conn = self.sentinel.master_for(self.service_name, ssl=ssl)


if Is_Cluster(conn):
conn = Cluster_Conn(
conn,
Expand All @@ -138,6 +140,7 @@
url,
address_remap,
)
self.cluster_flag = True

Check warning on line 143 in falkordb/falkordb.py

View check run for this annotation

Codecov / codecov/patch

falkordb/falkordb.py#L143

Added line #L143 was not covered by tests

self.connection = conn
self.flushdb = conn.flushdb
Expand Down Expand Up @@ -225,6 +228,22 @@

return self.connection.execute_command(CONFIG_CMD, "GET", name)[1]


def get_replica_connections(self):
#decide if its Sentinel or cluster
redis_mode= self.connection.execute_command("info")['redis_mode']
if redis_mode == "standalone":
replica_hostnames = self.sentinel.discover_slaves(service_name=self.service_name)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Update deprecated Redis terminology

The code uses deprecated 'slave' terminology. Redis has moved to more inclusive language.

Update the terminology:

-            replica_hostnames = self.sentinel.discover_slaves(service_name=self.service_name)
+            replica_hostnames = self.sentinel.discover_replicas(service_name=self.service_name)

-            return [(flag['hostname'], int(ip_port.split(':')[1])) for ip_port, flag in data.items() if 'slave' in flag["flags"] and flag["hostname"]]
+            return [(flag.get('hostname', ip_port.split(':')[0]), int(ip_port.split(':')[1])) 
+                    for ip_port, flag in data.items() 
+                    if 'replica' in flag.get('flags', [])]

Also applies to: 255-255

result = [(host,port) for host, port in replica_hostnames]
return result
elif redis_mode == "cluster":
data = self.connection.cluster_nodes()

Check warning on line 240 in falkordb/falkordb.py

View check run for this annotation

Codecov / codecov/patch

falkordb/falkordb.py#L234-L240

Added lines #L234 - L240 were not covered by tests
# List comprehension to get a list of (ip, port, hostname) tuples
host_port_list = [(ip_port.split(':')[0], ip_port.split(':')[1], flag['hostname']) for ip_port, flag in data.items() if 'slave' in flag["flags"]]
result = [tup for tup in host_port_list]
return result

Check warning on line 244 in falkordb/falkordb.py

View check run for this annotation

Codecov / codecov/patch

falkordb/falkordb.py#L242-L244

Added lines #L242 - L244 were not covered by tests
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Fix several critical issues in implementation

The method has multiple issues that need to be addressed:

  1. No error handling for invalid redis_mode
  2. Uses deprecated 'slave' terminology
  3. Potential KeyError when accessing flag['hostname']
  4. Inconsistent spacing around operators

Apply this diff to fix the issues:

-        redis_mode= self.connection.execute_command("info")['redis_mode']
+        redis_mode = self.connection.execute_command("info", section="server").get('redis_mode')
+        if not redis_mode:
+            raise ConnectionError("Unable to determine Redis mode")
+
         if redis_mode == "standalone":
-            replica_hostnames = self.sentinel.discover_slaves(service_name=self.service_name)
+            replica_hostnames = self.sentinel.discover_replicas(service_name=self.service_name)
             result = [(host,port) for host, port in replica_hostnames]
             return result
         elif redis_mode == "cluster":
             data = self.connection.cluster_nodes()
-            # List comprehension to get a list of (ip, port, hostname) tuples
-            host_port_list = [(ip_port.split(':')[0], ip_port.split(':')[1], flag['hostname']) for ip_port, flag in data.items() if 'slave' in flag["flags"]]
-            result = [tup for tup in host_port_list]
+            result = []
+            for ip_port, flag in data.items():
+                if 'replica' in flag.get('flags', []):  # Using 'replica' instead of 'slave'
+                    host, port = ip_port.split(':')
+                    result.append((host, int(port)))  # Convert port to int for consistency
+            return result
-            
+        else:
+            raise ValueError(f"Unsupported Redis mode: {redis_mode}")
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
redis_mode= self.connection.execute_command("info")['redis_mode']
if redis_mode == "standalone":
replica_hostnames = self.sentinel.discover_slaves(service_name=self.service_name)
result = [(host,port) for host, port in replica_hostnames]
return result
elif redis_mode == "cluster":
data = self.connection.cluster_nodes()
# List comprehension to get a list of (ip, port, hostname) tuples
host_port_list = [(ip_port.split(':')[0], ip_port.split(':')[1], flag['hostname']) for ip_port, flag in data.items() if 'slave' in flag["flags"]]
result = [tup for tup in host_port_list]
return result
redis_mode = self.connection.execute_command("info", section="server").get('redis_mode')
if not redis_mode:
raise ConnectionError("Unable to determine Redis mode")
if redis_mode == "standalone":
replica_hostnames = self.sentinel.discover_replicas(service_name=self.service_name)
result = [(host,port) for host, port in replica_hostnames]
return result
elif redis_mode == "cluster":
data = self.connection.cluster_nodes()
result = []
for ip_port, flag in data.items():
if 'replica' in flag.get('flags', []): # Using 'replica' instead of 'slave'
host, port = ip_port.split(':')
result.append((host, int(port))) # Convert port to int for consistency
return result
else:
raise ValueError(f"Unsupported Redis mode: {redis_mode}")
🧰 Tools
🪛 GitHub Check: codecov/patch

[warning] 233-239: falkordb/falkordb.py#L233-L239
Added lines #L233 - L239 were not covered by tests


[warning] 241-243: falkordb/falkordb.py#L241-L243
Added lines #L241 - L243 were not covered by tests



def config_set(self, name: str, value=None) -> None:
"""
Update a DB level configuration.
Expand Down