Hi Team,
I'm fetching couple of details from anypoint platform using rest apis ( ~10k api call) in parallel for-each. i'm aggregating the payload and inserting these records into MySql DB.
while executing i'm seeing below message in log and app is taking bit long time to execute. Just thinking is there a way to enable streaming in this flow.
flow:
-> app -> call platform rest api (parallel foreach, size 20) -> aggregate payload -> bulk insert to DB (wraped on for-each, size 1000)
INFO 2023-11-06 16:31:25,100 [[MuleRuntime].uber.35: [cloudhub-api-mgmt].api-mgmt-mainflow.BLOCKING @98f571b] [processor: bulk-upsert-api-to-db/processors/0/processors/0/errorHandler/0/processors/1; event: ccc94a30-7c91-11ee-bafa-bcd07412b57e] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: [TypedValue[value: 'org.mule.runtime.core.internal.streaming.bytes.ManagedCursorStreamProvider@72a18227', dataType: 'SimpleDataType{type=org.mule.runtime.core.internal.streaming.bytes.ManagedCursorStreamProvider, mimeType='application/json; charset=UTF-8'}'], TypedValue[value: 'org.mule.runtime.core.internal.streaming.bytes.ManagedCursorStreamProvider@926a8f3', dataType: 'SimpleDataType{type=org.mule.runtime.core.internal.streaming.bytes.ManagedCursorStreamProvider, mimeType='application/json; charset=UTF-8'}'], TypedValue[value: 'org.mule.runtime.core.internal.streaming.bytes.ManagedCursorStreamProvider@4f575f03', dataType: 'SimpleDataType{type=org.mule.runtime.core.internal.streaming.bytes.ManagedCursorStreamProvider, mimeType='application/json; charset=UTF-8'}'], TypedValue[value: 'org.mule.runtime.core.internal.streaming.bytes.ManagedCursorStreamProvider@75a51fa2', dataType: 'SimpleDataType{type=org.mule.runtime.core.internal.streaming.bytes.ManagedCursorStreamProvider, mimeType='application/json; charset=UTF-8'}'], TypedValue[value: 'org.mule.runtime.core.internal.streaming.bytes.ManagedCursorStreamProvider@beb0b8b', dataType: 'SimpleDataType{type=org.mule.runtime.core.internal.streaming.bytes.ManagedCursorStreamProvider, mimeType='application/json; charset=UTF-8'}'], TypedValue[value: 'org.mule.runtime.core.internal.streaming.bytes.ManagedCursorStreamProvider@37b65b35', dataType: 'SimpleDataType{type=org.mule.runtime.core.internal.streaming.bytes.ManagedCursorStreamProvider, mimeType='application/json; charset=UTF-8'}'], TypedValue[value: 'org.mule.runtime.core.internal.streaming.bytes.ManagedCursorStreamProvider@426cb587', dataType: 'SimpleDataType{type=org.mule.runtime.core.internal.streaming.bytes.ManagedCursorStreamProvider, mimeType='application/json; charset=UTF-8'}'], TypedValue[value: 'org.mule.runtime.core.internal.streaming.bytes.ManagedCursorStreamProvider@1fe9ea67', dataType: 'SimpleDataType{type=org.mule.runtime.core.internal.streaming.bytes.ManagedCursorStreamProvider, mimeType='application/json; charset=UTF-8'}'], TypedValue[value: 'org.mule.runtime.core.internal.streaming.bytes.ManagedCursorStreamProvider@4c0d1d42', dataType: 'SimpleDataType{type=org.mule.runtime.core.internal.streaming.bytes.ManagedCursorStreamProvider, mimeType='application/json; charset=UTF-8'}'], TypedValue[value: 'org.mule.runtime.core.internal.streaming.bytes.ManagedCursorStreamProvider@49e79805', dataType: 'SimpleDataType{type=org.mule.runtime.core.internal.streaming.bytes.ManagedCursorStreamProvider, mimeType='application/json; charset=UTF-8'}'], TypedValue[value: 'org.mulappreciate any suggestions.
2 respostas
@Shyam Raj Prasad i'm using bulk insert only inside for-each (batch size 1000).