Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 4.0 C2D_V2 ( Includes C2D V2 breaking changes + Free compute) #1912

Open
wants to merge 58 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
f6057f3
add datasets on ComputeAsset, new start compute fn
paulo-ocean Oct 31, 2024
9e6ec4d
fix type mismatch new c2d v2
paulo-ocean Oct 31, 2024
50e3ed8
fix tests
paulo-ocean Oct 31, 2024
0f0ad3e
fix tests, keep backwards compatibility due to tests running with exi…
paulo-ocean Oct 31, 2024
a9a099d
support for starting a free env compute job
paulo-ocean Nov 4, 2024
2107197
minor fix
paulo-ocean Nov 5, 2024
583db8e
add file object types
paulo-ocean Nov 6, 2024
7df390a
enforce asset files object type
paulo-ocean Nov 7, 2024
e312ec7
support for consumer signature on initialize compute
paulo-ocean Nov 12, 2024
88ee8d3
update docs and tests, pass signer account, not signer address
paulo-ocean Nov 12, 2024
c31c0c3
Release 4.0.0-next.0
paulo-ocean Nov 12, 2024
2c631d4
Release 4.0.0-next.1
paulo-ocean Nov 12, 2024
1030ee5
remove noZerox usage on compute routes query string/params
paulo-ocean Jan 7, 2025
c1b8546
fix conflicts
paulo-ocean Jan 7, 2025
c7a5ffd
merge main, fix conflicts
paulo-ocean Jan 14, 2025
a9e5614
fix conflict
paulo-ocean Jan 14, 2025
309b781
fix conflict
paulo-ocean Jan 14, 2025
bfceb46
rm node 18 image ref
paulo-ocean Jan 14, 2025
bd56020
put back change, was 18 not 16, 18 is ok
paulo-ocean Jan 14, 2025
fcebea0
try fix build ci
paulo-ocean Jan 20, 2025
ac61d94
try skip the delete part (images were not built before?)
paulo-ocean Jan 20, 2025
7f604f6
add the delete images part back
paulo-ocean Jan 20, 2025
d5fbd1a
wip: updating the compute env spces
paulo-ocean Jan 20, 2025
b272ef6
wip: update specs and docs
paulo-ocean Jan 21, 2025
dad53c6
try add -f flag
paulo-ocean Jan 21, 2025
c5afd5a
added types docs
paulo-ocean Jan 21, 2025
cdc1ec6
add -f flag on ci, removal of docker images, avoid fail if img not pr…
paulo-ocean Jan 22, 2025
16c1b90
Merge branch 'release_4.0' into issue-1901-new-c2d-env-specs
paulo-ocean Jan 22, 2025
cca8092
merge main & fix conflicts
paulo-ocean Feb 17, 2025
05a6086
Merge branch 'main' into release_4.0_C2D_V2
ahmedraza118 Feb 18, 2025
38385ef
fix merge typo
paulo-ocean Feb 18, 2025
0f6d73a
Merge branch 'issue-1901-new-c2d-env-specs' into release_4.0_C2D_V2
paulo-ocean Feb 19, 2025
2b98d19
modify computeStart params
paulo-ocean Feb 19, 2025
9184f3e
more refactoring and add missing definitions on compute env
paulo-ocean Feb 19, 2025
aeb2d58
debug it
paulo-ocean Feb 19, 2025
507274c
debug 2
paulo-ocean Feb 19, 2025
a4d28af
check if provider supports free compute routes first
paulo-ocean Feb 19, 2025
691debb
fix typo
paulo-ocean Feb 19, 2025
b391463
fix/update getEnvs response
paulo-ocean Feb 20, 2025
82a288d
check for freeCompute route endpoint support first
paulo-ocean Feb 20, 2025
bd64c18
fix lint
paulo-ocean Feb 20, 2025
c109733
wip: still not all done
paulo-ocean Feb 20, 2025
3ee7907
another try
paulo-ocean Feb 20, 2025
6eccdb8
another try
paulo-ocean Feb 20, 2025
c7c31da
Merge 6eccdb8c0c54321deade91755aa59e991778a12c into c8314806269ba21bb…
paulo-ocean Feb 20, 2025
1b20453
Updating ComputeExamples.md
Feb 20, 2025
0490819
remove console.log
paulo-ocean Feb 21, 2025
5f1f29d
Merge branch 'release_4.0_C2D_V2' of github.com:oceanprotocol/ocean.j…
paulo-ocean Feb 21, 2025
9f2f49c
rebuild package-lock
paulo-ocean Feb 21, 2025
30a67e7
Merge branch 'main' into release_4.0_C2D_V2
paulo-ocean Feb 25, 2025
6f0371e
refactor, separate function for free compute start
paulo-ocean Feb 25, 2025
d1e476e
remove unused param
paulo-ocean Feb 25, 2025
1216dd7
Merge branch 'main' into release_4.0_C2D_V2
paulo-ocean Feb 25, 2025
9143f18
update signature for free c2d (did + nonce only)
paulo-ocean Feb 25, 2025
c6d8b98
yet another signature update, on nodes, message is only the nonce
paulo-ocean Feb 25, 2025
3b86e82
Merge branch 'main' into release_4.0_C2D_V2
paulo-ocean Feb 26, 2025
42d6422
wip: add streamable logs
paulo-ocean Feb 26, 2025
94afd48
add preliminary support for streamable logs
paulo-ocean Feb 27, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
208 changes: 118 additions & 90 deletions ComputeExamples.md
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,8 @@ import {
configHelperNetworks,
ConfigHelper,
getEventFromTx,
amountToUnits
amountToUnits,
isDefined
} from '@oceanprotocol/lib'
```

Expand Down Expand Up @@ -275,6 +276,9 @@ let resolvedAlgorithmDdo: DDO

let computeJobId: string
let agreementId: string

let computeRoutePath: string
let hasFreeComputeSupport: boolean
```

### 4.3 Helper methods
Expand Down Expand Up @@ -594,119 +598,143 @@ Now, let's check that we successfully published a algorithm (create NFT + Datato
let's check the free compute environment
```Typescript
const computeEnv = computeEnvs[resolvedDatasetDdo.chainId].find(
(ce) => ce.priceMin === 0
(ce) => ce.priceMin === 0 || isDefined(ce.free)
)
console.log('Free compute environment = ', computeEnv)
```
<!--
assert(computeEnv, 'Cannot find the free compute env')
-->

Let's have 5 minute of compute access
```Typescript
const mytime = new Date()
const computeMinutes = 5
mytime.setMinutes(mytime.getMinutes() + computeMinutes)
const computeValidUntil = Math.floor(mytime.getTime() / 1000)

const assets: ComputeAsset[] = [
{
documentId: resolvedDatasetDdo.id,
serviceId: resolvedDatasetDdo.services[0].id
computeRoutePath = await ProviderInstance.getComputeStartRoutes(providerUrl, true)
if (isDefined(computeRoutePath)) {
hasFreeComputeSupport = true
Let's have 5 minute of compute access
```Typescript
const mytime = new Date()
const computeMinutes = 5
mytime.setMinutes(mytime.getMinutes() + computeMinutes)
const computeValidUntil = Math.floor(mytime.getTime() / 1000)

const assets: ComputeAsset[] = [
{
documentId: resolvedDatasetDdo.id,
serviceId: resolvedDatasetDdo.services[0].id
}
]
const dtAddressArray = [resolvedDatasetDdo.services[0].datatokenAddress]
const algo: ComputeAlgorithm = {
documentId: resolvedAlgorithmDdo.id,
serviceId: resolvedAlgorithmDdo.services[0].id
}
]
const dtAddressArray = [resolvedDatasetDdo.services[0].datatokenAddress]
const algo: ComputeAlgorithm = {
documentId: resolvedAlgorithmDdo.id,
serviceId: resolvedAlgorithmDdo.services[0].id
}

const providerInitializeComputeResults = await ProviderInstance.initializeCompute(
assets,
algo,
computeEnv.id,
computeValidUntil,
providerUrl,
await consumerAccount.getAddress()
)
```
<!--
assert(!('error' in providerInitializeComputeResults), 'Cannot order algorithm')
-->
```Typescript
algo.transferTxId = await handleOrder(
providerInitializeComputeResults.algorithm,
resolvedAlgorithmDdo.services[0].datatokenAddress,
consumerAccount,
computeEnv.consumerAddress,
0
)
for (let i = 0; i < providerInitializeComputeResults.datasets.length; i++) {
assets[i].transferTxId = await handleOrder(
providerInitializeComputeResults.datasets[i],
dtAddressArray[i],
const providerInitializeComputeResults = await ProviderInstance.initializeCompute(
assets,
algo,
computeEnv.id,
computeValidUntil,
providerUrl,
consumerAccount
)
```
<!--
assert(!('error' in providerInitializeComputeResults), 'Cannot order algorithm')
-->
```Typescript
algo.transferTxId = await handleOrder(
providerInitializeComputeResults.algorithm,
resolvedAlgorithmDdo.services[0].datatokenAddress,
consumerAccount,
computeEnv.consumerAddress,
0
)
}
for (let i = 0; i < providerInitializeComputeResults.datasets.length; i++) {
assets[i].transferTxId = await handleOrder(
providerInitializeComputeResults.datasets[i],
dtAddressArray[i],
consumerAccount,
computeEnv.consumerAddress,
0
)
}

const computeJobs = await ProviderInstance.computeStart(
providerUrl,
consumerAccount,
computeEnv.id,
assets[0],
algo
)
const computeJobs = await ProviderInstance.freeComputeStart(
providerUrl,
consumerAccount,
computeEnv.id,
assets,
algo
)

```
<!--
assert(computeJobs, 'Cannot start compute job')
-->
Let's save the compute job it, we re going to use later
```Typescript
computeJobId = computeJobs[0].jobId
// eslint-disable-next-line prefer-destructuring
agreementId = computeJobs[0].agreementId
```
<!--
assert(computeJobs, 'Cannot start compute job')
-->
Let's save the compute job it, we re going to use later
```Typescript
computeJobId = computeJobs[0].jobId
// eslint-disable-next-line prefer-destructuring
agreementId = computeJobs[0].agreementId
} else {
assert(
computeRoutePath === null,
'Route path for free compute is not defined (perhaps because provider does not support it yet?)'
)
hasFreeComputeSupport = false
}

```

## 11. Check compute status and get download compute results URL
### 11.1 Check compute status
You can also add various delays so you see the various states of the compute job
```Typescript
const jobStatus = await ProviderInstance.computeStatus(
providerUrl,
await consumerAccount.getAddress(),
computeJobId,
agreementId
)
```
<!--
assert(jobStatus, 'Cannot retrieve compute status!')
-->
Now, let's see the current status of the previously started computer job
```Typescript
console.log('Current status of the compute job: ', jobStatus)
if (!hasFreeComputeSupport) {
assert(
computeRoutePath === null,
'Compute route path for free compute is not defined (perhaps because provider does not support it yet?)'
)
} else {
You can also add various delays so you see the various states of the compute job
```Typescript
const jobStatus = await ProviderInstance.computeStatus(
providerUrl,
await consumerAccount.getAddress(),
computeJobId,
agreementId
)
```
<!--
assert(jobStatus, 'Cannot retrieve compute status!')
-->
Now, let's see the current status of the previously started computer job
```Typescript
console.log('Current status of the compute job: ', jobStatus)
}

```

### 11.2 Get download compute results URL
```Typescript
await sleep(10000)
const downloadURL = await ProviderInstance.getComputeResultUrl(
providerUrl,
consumerAccount,
computeJobId,
0
)
```
<!--
assert(downloadURL, 'Provider getComputeResultUrl failed!')
-->
Let's check the compute results url for the specified index
```Typescript
console.log(`Compute results URL: ${downloadURL}`)
if (!hasFreeComputeSupport) {
assert(
computeRoutePath === null,
'Compute route path for free compute is not defined (perhaps because provider does not support it yet?)'
)
} else {
```Typescript
await sleep(10000)
const downloadURL = await ProviderInstance.getComputeResultUrl(
providerUrl,
consumerAccount,
computeJobId,
0
)
```
<!--
assert(downloadURL, 'Provider getComputeResultUrl failed!')
-->
Let's check the compute results url for the specified index
```Typescript
console.log(`Compute results URL: ${downloadURL}`)
}

```

Expand Down
44 changes: 38 additions & 6 deletions docs/classes/Provider.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ ___

### computeStart

▸ **computeStart**(`providerUri`, `consumer`, `computeEnv`, `dataset`, `algorithm`, `signal?`, `additionalDatasets?`, `output?`): `Promise`<[`ComputeJob`](../interfaces/ComputeJob.md) \| [`ComputeJob`](../interfaces/ComputeJob.md)[]\>
▸ **computeStart**(`providerUri`, `signer`, `computeEnv`, `datasets`, `algorithm`, `resources`, `chainId`, `output?`, `freeEnvironment`, `signal?`): `Promise`<[`ComputeJob`](../interfaces/ComputeJob.md) \| [`ComputeJob`](../interfaces/ComputeJob.md)[]\>

Instruct the provider to start a compute job

Expand All @@ -106,13 +106,15 @@ Instruct the provider to start a compute job
| Name | Type | Description |
| :------ | :------ | :------ |
| `providerUri` | `string` | The provider URI. |
| `consumer` | `Signer` | - |
| `signer` | `Signer` | - | The consumer signer/account
| `computeEnv` | `string` | The compute environment. |
| `dataset` | [`ComputeAsset`](../interfaces/ComputeAsset.md) | The dataset to start compute on |
| `datasets` | [`ComputeAsset`](../interfaces/ComputeAsset.md) | The dataset to start compute on |
| `algorithm` | [`ComputeAlgorithm`](../interfaces/ComputeAlgorithm.md) | The algorithm to start compute with. |
| `signal?` | `AbortSignal` | abort signal |
| `additionalDatasets?` | [`ComputeAsset`](../interfaces/ComputeAsset.md)[] | The additional datasets if that is the case. |
| `resources` | [`ComputeResourceRequest`](../interfaces/ComputeResourcesRequest.md) | The resources to start compute with. |
| `chainId?` | [`number`] | The network for the payments |
| `output?` | [`ComputeOutput`](../interfaces/ComputeOutput.md) | The compute job output settings. |
| `signal?` | `AbortSignal` | abort signal |


#### Returns

Expand Down Expand Up @@ -486,7 +488,7 @@ Initializes the provider for a compute request.
| `computeEnv` | `string` | The compute environment. |
| `validUntil` | `number` | The job expiration date. |
| `providerUri` | `string` | The provider URI. |
| `accountId` | `string` | caller address |
| `signer` | `Signer` | caller account |
| `signal?` | `AbortSignal` | abort signal |

#### Returns
Expand All @@ -501,6 +503,36 @@ ProviderComputeInitialize data

___


### computeStreamableLogs

▸ **computeStreamableLogs**(`providerUri`, `signer`, `jobId`, `signal?`): `Promise`<`any`\>

Gets the streamable compute logs.

#### Parameters

| Name | Type | Description |
| :------ | :------ | :------ |
| `providerUri` | `string` | The provider URI. |
| `signer` | `Signer` | The signer. |
| `jobId` | `string` | The Job Id. |
| `signal?` | `AbortSignal` | The abort signal. |


#### Returns

`Promise`<`any`\>

The compute logs.

#### Defined in

[services/Provider.ts:908](https://github.com/oceanprotocol/ocean.js/blob/c99bc5c6/src/services/Provider.ts#L908)

___


### inputMatch

▸ `Private` **inputMatch**(`input`, `regexp`, `conversorName`): `Object`
Expand Down
30 changes: 30 additions & 0 deletions docs/interfaces/ComputeEnvFees.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
[@oceanprotocol/lib](../README.md) / [Exports](../modules.md) / ComputeEnvFees

# Interface: ComputeEnvFees

## Table of contents

### Properties

- [feeToken](ComputeEnvFees.md#feeToken)
- [prices](ComputeEnvFees.md#prices)

## Properties

### feeToken

• **feeToken**: `string`

#### Defined in

[@types/Compute.ts:42](https://github.com/oceanprotocol/ocean.js/blob/c99bc5c6/src/@types/Compute.ts#L42)

___

### prices

• **prices**: `ComputeResourcesPricingInfo`[]

#### Defined in

[@types/Compute.ts:43](https://github.com/oceanprotocol/ocean.js/blob/c99bc5c6/src/@types/Compute.ts#L43)
20 changes: 20 additions & 0 deletions docs/interfaces/ComputeEnvFeesStructure.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
[@oceanprotocol/lib](../README.md) / [Exports](../modules.md) / ComputeEnvFeesStructure

# Interface: ComputeEnvFeesStructure

## Table of contents

### Properties

- [feeToken](ComputeEnvFeesStructure.md#chainId)

## Properties

### chainId

• **chainId**: `ComputeEnvFees`

#### Defined in

[@types/Compute.ts:42](https://github.com/oceanprotocol/ocean.js/blob/c99bc5c6/src/@types/Compute.ts#L46)

Loading