Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for new Vector type #74

Merged
merged 4 commits into from
Oct 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/maven.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ jobs:
# Label used to access the service container
falkordb:
# Docker Hub image
image: falkordb/falkordb:latest
image: falkordb/falkordb:edge
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

Issues Detected with Using the 'edge' Tag for FalkorDB

The 'edge' version of FalkorDB is actively maintained with recent commits. However, there are multiple open issues that may affect the stability of your CI/CD pipeline:

  • Recent Open Issues:
    • Delete arc specific docker tags from dockerhub
    • flushdb bug?
    • Unexpected Exception Raised "Type mismatch: expected Integer but was Null"
    • Index not being used for some Match queries.

Recommendations:

  1. Use a Specific Version Tag: For more predictable and stable builds, consider specifying a particular version of FalkorDB instead of using the 'edge' tag.
  2. Monitor Open Issues: If you choose to continue with the 'edge' tag, regularly review and address the open issues to mitigate potential risks.
  3. Compatibility Check: Ensure that your codebase is compatible with the changes and updates in the 'edge' version of FalkorDB.
🔗 Analysis chain

Consider the implications of using the 'edge' tag for FalkorDB

Changing the FalkorDB image from 'latest' to 'edge' allows testing against the most recent development version, which may include new features or bug fixes. However, this comes with potential risks:

  1. 'edge' versions are typically less stable and may introduce breaking changes.
  2. This could lead to inconsistent or failing tests if incompatibilities are introduced.

Consider the following recommendations:

  1. Ensure that your codebase is compatible with any changes in the 'edge' version of FalkorDB.
  2. For more predictable and reproducible builds, consider using a specific version tag instead of 'edge'.
  3. If you proceed with 'edge', implement a process to regularly review and update your code to accommodate FalkorDB changes.

To verify the impact of this change, you can run the following script:

This script will help you assess any recent changes or potential issues with the FalkorDB edge version that might affect your workflow.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check for any recent changes or issues with the FalkorDB edge version

# Check the latest commits and issues in the FalkorDB repository
echo "Recent commits in FalkorDB repository:"
gh repo view falkordb/falkordb --json updatedAt,defaultBranchRef --jq '.defaultBranchRef.name as $branch | "Latest commit on \($branch): \(.updatedAt)"'

echo -e "\nRecent issues in FalkorDB repository:"
gh issue list --repo falkordb/falkordb --limit 5 --state all

# Check if there are any open pull requests that might affect the edge version
echo -e "\nOpen pull requests that might affect the edge version:"
gh pr list --repo falkordb/falkordb --state open --limit 5

Length of output: 1641

# Map port 6379 on the Docker host to port 6379 on the FalkorDB container
ports:
- 6379:6379
Expand Down
3 changes: 3 additions & 0 deletions .sdkmanrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Enable auto-env through the sdkman_auto_env config
# Add key=value pairs of SDKs to use below
java=17.0.12-graal
21 changes: 19 additions & 2 deletions src/main/java/com/falkordb/impl/resultset/ResultSetImpl.java
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
import redis.clients.jedis.exceptions.JedisDataException;

import java.util.*;
import java.util.stream.Collectors;

public class ResultSetImpl implements ResultSet {

Expand All @@ -25,7 +26,7 @@
/**
* @param rawResponse the raw representation of response is at most 3 lists of
* objects. The last list is the statistics list.
* @param graph the graph connection
* @param graph the graph connection
* @param cache the graph local cache
*/
@SuppressWarnings("unchecked")
Expand Down Expand Up @@ -157,7 +158,7 @@

/**
* @param graphEntity graph entity
* @param id entity id to be set to the graph entity
* @param id entity id to be set to the graph entity
*/
private void deserializeGraphEntityId(GraphEntity graphEntity, long id) {
graphEntity.setId(id);
Expand Down Expand Up @@ -246,6 +247,8 @@
return deserializeMap(obj);
case VALUE_POINT:
return deserializePoint(obj);
case VALUE_VECTORF32:
return deserializeVector(obj);
case VALUE_UNKNOWN:
default:
return obj;
Expand All @@ -256,6 +259,20 @@
return new Point(BuilderFactory.DOUBLE_LIST.build(rawScalarData));
}

private List<Float> deserializeVector(Object rawScalarData) {
List<byte[]> array = (List<byte[]>) rawScalarData;
return array.stream()
.map(val -> {
try {
return Float.parseFloat(SafeEncoder.encode(val));
} catch (NumberFormatException e) {

Check warning on line 268 in src/main/java/com/falkordb/impl/resultset/ResultSetImpl.java

View check run for this annotation

Codecov / codecov/patch

src/main/java/com/falkordb/impl/resultset/ResultSetImpl.java#L268

Added line #L268 was not covered by tests
// Handle the exception appropriately
throw new GraphException("Invalid float value in vector data", e);

Check warning on line 270 in src/main/java/com/falkordb/impl/resultset/ResultSetImpl.java

View check run for this annotation

Codecov / codecov/patch

src/main/java/com/falkordb/impl/resultset/ResultSetImpl.java#L270

Added line #L270 was not covered by tests
}
})
.collect(Collectors.toList());
}

@SuppressWarnings("unchecked")
private Map<String, Object> deserializeMap(Object rawScalarData) {
List<Object> keyTypeValueEntries = (List<Object>) rawScalarData;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@ enum ResultSetScalarTypes {
VALUE_NODE,
VALUE_PATH,
VALUE_MAP,
VALUE_POINT;
VALUE_POINT,
VALUE_VECTORF32;
gkorland marked this conversation as resolved.
Show resolved Hide resolved

private static final ResultSetScalarTypes[] values = values();

Expand Down
13 changes: 13 additions & 0 deletions src/test/java/com/falkordb/GraphAPITest.java
Original file line number Diff line number Diff line change
Expand Up @@ -798,6 +798,19 @@ public void test64bitnumber() {
Assert.assertEquals(Long.valueOf(value), r.getValue(0));
}

@Test
public void testVecf32() {
ResultSet resultSet = client.query("RETURN vecf32([2.1, -0.82, 1.3, 4.5]) AS vector");
Assert.assertEquals(1, resultSet.size());
Record r = resultSet.iterator().next();
List<Float> vector = r.getValue(0);
Assert.assertEquals(4, vector.size());
Assert.assertEquals(2.1f, vector.get(0), 0.01);
Assert.assertEquals(-0.82f, vector.get(1), 0.01);
Assert.assertEquals(1.3f, vector.get(2), 0.01);
Assert.assertEquals(4.5f, vector.get(3), 0.01);
}

@Test
public void testCachedExecution() {
client.query("CREATE (:N {val:1}), (:N {val:2})");
Expand Down