Update ingest/change_compactor with extended Change struct

[chowbao]

Related to #5535

The change_compactor should be updated to use the new fields in the Change struct

New fields:

• Reason
• OperationIndex
• Transaction
• Ledger
• LedgerUpgrade

None of the existing Type, Pre, Post field logic needs to be changed.

The compacted changes from AddChange(change Change) should take the most recent fields from above if there is already an existing change for c.cache[ledgerKeyString]

For example in https://github.com/stellar/go/blob/master/ingest/change_compactor.go#L152-L166

	switch existingChange.LedgerEntryChangeType() {
	case xdr.LedgerEntryChangeTypeLedgerEntryCreated:
		// If existing type is created it means that this entry does not
		// exist in a DB so we update entry change.
		c.cache[ledgerKeyString] = Change{
			Type: key.Type,
			Pre:  existingChange.Pre, // = nil
			Post: change.Post,
			Reason: change.Reason,
			OperationIndex: change.OperationIndex,
			Transaction: change.Transaction,
			Ledger: change.Ledger,
			LedgerUpgrade: change.LedgerUpgrade,
		}

Alternatively instead of propagating the latest Change.Reason/OperationIndex/etc... to c.cache[ledgerKeyString] = Change{} it is possible to keep the existingChange.Reason/OperationIndex/etc... instead of change.Reason/OperationIndex/etc.... This would be the less preferred option from stellar-etl

[urvisavla]

@chowbao It would be helpful for my understanding if you could explain how these new fields can be used (with an example?). Thanks!

[chowbao]

@chowbao It would be helpful for my understanding if you could explain how these new fields can be used (with an example?). Thanks!

Of course

So in stellar-etl we would use the changes to add new columns to some of our tables. For example, our contract_data state table that records the compacted ledger entry changes for contract_data. New columns that would be helpful for us to add would be like the transaction_hash or ledger_sequence that the compacted change occurred. This is nice because now we wouldn't have to manually pass the LedgerCloseMeta as we process changes (Ledger already in the change).

stellar-etl currently has this flow in the code if that helps clear things up as well

• Change compact the changes
• Transform/process relevant contract data changes
• Pull information from the LCM header passed in the function instead of being in the function itself
• Output and upload to data lake to ingest into BigQuery (Hubble)

By also passing new fields like OperationIndex and Transaction we can also reasonable add new columns to the contract_data table like transaction_hash or operation_id that is tied to the change

[chowbao]

We will copy and modify the changeCompactor for stellar-etl only within go/stellar-etl

The changeCompactor within the stellar/go/ingest will not be changed