Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added get_replica_connections function #72

Open
wants to merge 39 commits into
base: main
Choose a base branch
from
Open

added get_replica_connections function #72

Changes from 7 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
e7beff6
added get_replica_connections function
MuhammadQadora Oct 28, 2024
6a717ab
only return connections
MuhammadQadora Nov 6, 2024
0791c7e
removed flag
MuhammadQadora Nov 14, 2024
c77580e
updated to suite Coderabbit comments
MuhammadQadora Nov 14, 2024
5ee68b7
fixed function
MuhammadQadora Nov 14, 2024
c4dc33a
removed unnecessary variable
MuhammadQadora Nov 14, 2024
2c20a1e
Update falkordb.py
swilly22 Nov 14, 2024
99b6671
added error handling
MuhammadQadora Nov 14, 2024
dd8a4ad
removed error handler
MuhammadQadora Nov 14, 2024
8eed87f
added test
MuhammadQadora Nov 18, 2024
e646320
added tests
MuhammadQadora Nov 20, 2024
4a3d585
fixed path
MuhammadQadora Nov 20, 2024
a1262e0
updated
MuhammadQadora Nov 20, 2024
199a1e5
updated
MuhammadQadora Nov 20, 2024
71622fe
updated test
MuhammadQadora Nov 21, 2024
9f261fc
test for clusterCon and SentinelCon
MuhammadQadora Nov 21, 2024
9d21e95
removed wrong import
MuhammadQadora Nov 21, 2024
6aad7bd
fixed test
MuhammadQadora Nov 21, 2024
2a96be0
updated test
MuhammadQadora Nov 22, 2024
b693987
changed
MuhammadQadora Nov 24, 2024
519b9dd
updated
MuhammadQadora Nov 24, 2024
557f04f
added compose files
MuhammadQadora Dec 2, 2024
222cfb4
created deployments standalone, redis cluster and redis replication f…
MuhammadQadora Dec 16, 2024
a9f597e
testing
MuhammadQadora Dec 18, 2024
ac8471e
created deployments standalone, redis cluster and redis replication f…
MuhammadQadora Dec 19, 2024
cfd0864
created deployments standalone, redis cluster and redis replication f…
MuhammadQadora Dec 19, 2024
65e41d0
full path
MuhammadQadora Dec 19, 2024
d112322
removed exceptions
MuhammadQadora Dec 19, 2024
ad7cedd
moved docker files to docker folder
swilly22 Dec 22, 2024
ac4cfa4
changed image to use falkordb/falkordb:edge and overriden the entrypoint
MuhammadQadora Dec 26, 2024
5a444db
fixed entrypoint
MuhammadQadora Dec 26, 2024
4cc0b6a
added yml suffix
MuhammadQadora Dec 26, 2024
51d4d90
updated image tag
MuhammadQadora Dec 29, 2024
cb8cd2e
[WIP] improved over all test latency
swilly22 Dec 29, 2024
1ad3162
Merge branch 'main' of https://github.com/FalkorDB/falkordb-py into 7…
MuhammadQadora Dec 30, 2024
4d4c1ac
change compose files and removed the need for docker compose down
MuhammadQadora Jan 1, 2025
263e8bf
change compose files and removed the need for docker compose down
MuhammadQadora Jan 2, 2025
af4f748
removed echos
MuhammadQadora Jan 2, 2025
b1d6c41
enhanced waiting
MuhammadQadora Jan 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 33 additions & 1 deletion falkordb/falkordb.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@
dynamic_startup_nodes=True,
url=None,
address_remap=None,
decode_responses=True
):

conn = redis.Redis(
Expand All @@ -92,7 +93,7 @@
encoding_errors=encoding_errors,
charset=charset,
errors=errors,
decode_responses=True,
decode_responses=decode_responses,
retry_on_timeout=retry_on_timeout,
retry_on_error=retry_on_error,
ssl=ssl,
Expand Down Expand Up @@ -125,6 +126,7 @@
self.sentinel, self.service_name = Sentinel_Conn(conn, ssl)
conn = self.sentinel.master_for(self.service_name, ssl=ssl)


if Is_Cluster(conn):
conn = Cluster_Conn(
conn,
Expand Down Expand Up @@ -225,6 +227,36 @@

return self.connection.execute_command(CONFIG_CMD, "GET", name)[1]


def get_replica_connections(self):
"""
Retrieve a list of connections for FalkorDB replicas.

This function determines the FalkorDB setup (Sentinel or Cluster) and returns
the hostnames and ports of the replicas.

Returns:
list of tuple: A list of (hostname, port) tuples representing the
replica connections.

Raises:
ConnectionError: If unable to connect or retrieve information from
the FalkorDB setup.
ValueError: If the `mode` is neither Sentinel nor Cluster.
"""
# decide if it's Sentinel or cluster
mode = self.connection.execute_command("info")['redis_mode']
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Add proper error handling

The code lacks proper error handling for Redis commands and dictionary access.

Add comprehensive error handling:

-        mode = self.connection.execute_command("info")['redis_mode']
+        try:
+            info = self.connection.execute_command("info", section="server")
+            mode = info.get('redis_mode')
+            if not mode:
+                raise ConnectionError("Unable to determine Redis mode")
+        except redis.RedisError as e:
+            raise ConnectionError("Failed to retrieve Redis mode") from e

         elif mode == "cluster":
-            data = self.connection.cluster_nodes()
-            return [(flag['hostname'], int(ip_port.split(':')[1])) for ip_port, flag in data.items() if 'slave' in flag["flags"] and flag["hostname"]]
+            try:
+                data = self.connection.cluster_nodes()
+                result = []
+                for ip_port, flag in data.items():
+                    if 'replica' in flag.get('flags', []):
+                        try:
+                            host = flag.get('hostname', ip_port.split(':')[0])
+                            port = int(ip_port.split(':')[1])
+                            result.append((host, port))
+                        except (IndexError, ValueError):
+                            continue  # Skip malformed entries
+                return result
+            except redis.RedisError as e:
+                raise ConnectionError("Failed to retrieve cluster nodes") from e

Also applies to: 254-255

🧰 Tools
🪛 GitHub Check: codecov/patch

[warning] 249-255: falkordb/falkordb.py#L249-L255
Added lines #L249 - L255 were not covered by tests

Copy link
Preview

Copilot AI Nov 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ensure that 'redis_mode' exists in the dictionary returned by the 'info' command to avoid a KeyError.

Suggested change
mode = self.connection.execute_command("info")['redis_mode']
mode = self.connection.execute_command("info").get('redis_mode', None)

Copilot is powered by AI, so mistakes are possible. Review output carefully before use.

Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options
if hasattr(self, 'sentinel') and self.sentinel is not None:
replica_hostnames = self.sentinel.discover_slaves(service_name=self.service_name)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Update deprecated Redis terminology

The code uses deprecated 'slave' terminology. Redis has moved to more inclusive language.

Update the terminology:

-            replica_hostnames = self.sentinel.discover_slaves(service_name=self.service_name)
+            replica_hostnames = self.sentinel.discover_replicas(service_name=self.service_name)

-            return [(flag['hostname'], int(ip_port.split(':')[1])) for ip_port, flag in data.items() if 'slave' in flag["flags"] and flag["hostname"]]
+            return [(flag.get('hostname', ip_port.split(':')[0]), int(ip_port.split(':')[1])) 
+                    for ip_port, flag in data.items() 
+                    if 'replica' in flag.get('flags', [])]

Also applies to: 255-255

return [(host, port) for host, port in replica_hostnames]
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Update implementation to use modern Redis terminology and handle edge cases

The current implementation has several issues:

  1. Uses deprecated 'slave' terminology
  2. Assumes 'hostname' exists in cluster node flags
  3. Doesn't convert port to integer
-            replica_hostnames = self.sentinel.discover_slaves(service_name=self.service_name)
-            return [(host, port) for host, port in replica_hostnames]
+            try:
+                replica_hostnames = self.sentinel.discover_replicas(service_name=self.service_name)
+                return [(host, int(port)) for host, port in replica_hostnames]
+            except Exception as e:
+                raise ConnectionError(f"Failed to discover replicas: {str(e)}")

# For cluster mode:
-            return [(flag['hostname'], ip_port.split(':')[1]) for ip_port, flag in data.items() if 'slave' in flag["flags"]]
+            result = []
+            for ip_port, flag in data.items():
+                if 'replica' in flag.get('flags', []):
+                    try:
+                        host = ip_port.split(':')[0]  # Use IP as hostname
+                        port = int(ip_port.split(':')[1])
+                        result.append((host, port))
+                    except (IndexError, ValueError) as e:
+                        continue  # Skip malformed entries
+            return result

Also applies to: 254-255

elif mode == "cluster":
data = self.connection.cluster_nodes()
return [(flag['hostname'], ip_port.split(':')[1]) for ip_port, flag in data.items() if 'slave' in flag["flags"]]

Check warning on line 254 in falkordb/falkordb.py

View check run for this annotation

Codecov / codecov/patch

falkordb/falkordb.py#L248-L254 

Added lines #L248 - L254 were not covered by tests
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Fix potential KeyError in cluster mode implementation

The current implementation assumes the presence of a 'hostname' key and doesn't handle potential errors.

Apply this diff:

-            return [(flag['hostname'], ip_port.split(':')[1]) for ip_port, flag in data.items() if 'slave' in flag["flags"]]
+            result = []
+            for ip_port, flag in data.items():
+                if 'replica' in flag.get('flags', []):  # Using 'replica' instead of 'slave'
+                    try:
+                        host = ip_port.split(':')[0]  # Use IP as hostname
+                        port = int(ip_port.split(':')[1])
+                        result.append((host, port))
+                    except (IndexError, ValueError) as e:
+                        continue  # Skip malformed entries
+            return result
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
return [(flag['hostname'], ip_port.split(':')[1]) for ip_port, flag in data.items() if 'slave' in flag["flags"]]
result = []
for ip_port, flag in data.items():
if 'replica' in flag.get('flags', []): # Using 'replica' instead of 'slave'
try:
host = ip_port.split(':')[0] # Use IP as hostname
port = int(ip_port.split(':')[1])
result.append((host, port))
except (IndexError, ValueError) as e:
continue # Skip malformed entries
return result
🧰 Tools
🪛 GitHub Check: codecov/patch

[warning] 248-254: falkordb/falkordb.py#L248-L254
Added lines #L248 - L254 were not covered by tests

else:
raise ValueError(f"Unsupported Redis mode: {redis_mode}")

Check warning on line 256 in falkordb/falkordb.py

View check run for this annotation

Codecov / codecov/patch

falkordb/falkordb.py#L256 

Added line #L256 was not covered by tests
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Fix undefined variable reference

The error message references redis_mode but the variable is named mode.

Apply this diff:

-            raise ValueError(f"Unsupported Redis mode: {redis_mode}")
+            raise ValueError(f"Unsupported Redis mode: {mode}")
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
raise ValueError(f"Unsupported Redis mode: {redis_mode}")
raise ValueError(f"Unsupported Redis mode: {mode}")
🧰 Tools
🪛 GitHub Check: codecov/patch

[warning] 256-256: falkordb/falkordb.py#L256
Added line #L256 was not covered by tests

🪛 Ruff

256-256: redis_mode may be undefined, or defined from star imports

(F405)




def config_set(self, name: str, value=None) -> None:
"""
Update a DB level configuration.
Expand Down