Deleting Interactions: Example 02

An Reltio Data Science example showing delete interactions.

Consider the dfEvents Dataset with the following schema:

|-- Id: string (nullable = true)
|-- members: struct (nullable = true)
| |-- HCO: array (nullable = true)
| | |-- element: string (containsNull = true)
| |-- HCP: array (nullable = true)
| | |-- element: string (containsNull = true)

Using Spark SQL we can get Ids of the interactions we are going to delete:

val deleteIn = dfEvents.select("Id", "members.HCO", "members.HCP").filter("HCP[0]['Id'] is null")

To delete IDs from the Dataset:

iConnection.dataDelete
  .withBuilder(
    new DeleteInteractionBuilder()
      .fromDataFrame(deleteIn)
      .idColumn(s "$p_idCol")
      .ofType(s "configuration/interactionsType/$p_interactionType")
  )
  .build
.deleteAndGetJobId()