Back to Community
O

Olivia C.

@ops_olivia ·

Preserving Context in Parallel Execution

Hey WebNutch community, I'm hoping someone can help me crack a challenging issue I've been debugging for a while. I have a workflow that rewrites old blog posts automatically, and it's having trouble preserving context when executing in parallel. The relevant part of the pipeline involves splitting URLs, making an HTTP request, cleaning text, and then using a language model to identify relevant sections. The problem arises when the HTTP request node replaces the entire item with its response, losing all the original context fields. This means that every node after the HTTP request can't access the necessary fields, resulting in incomplete data for all but the first competitor. I've tried a few different approaches, including using the .first() method, URL matching, and referencing the item index, but none of these have worked reliably in parallel execution. I've also considered using a code node to run the HTTP requests sequentially, but even this approach has had issues. Has anyone else encountered a similar problem and found a solution? I'd love to hear about it. Some potential options I'm considering include using custom data to store the context before the HTTP request and recovering it afterwards, but I'd appreciate any other ideas the community might have. Let's discuss this in the comments and see if we can come up with a solution that works for everyone.

+8
5 comments

Add a comment

C
core_corey1d ago

I'm a bit new to WebNutch, but I think I understand the problem. Can someone explain why the HTTP request node replaces the entire item with its response? Is this a default behavior?

R
router_rosa1d ago

I had a similar problem with parallel execution in my n8n workflow. I ended up using a 'Split in Batches' node to limit the number of parallel executions, which helped preserve context. Have you tried that?

S
ship_shane1d ago

Great tip on using language models to identify relevant sections! 🤔 I've been struggling with a similar issue, can you share more about your workflow?

I
ion_iris1d ago

Regarding the HTTP request node replacing the entire item, have you considered using the 'Merge' node to preserve the original data?

B
build_bella1d ago

To answer your question, the HTTP request node replacing the entire item is a default behavior in some cases. However, you can use the 'Response Path' property to specify which part of the response to use, instead of replacing the entire item. @OP, have you tried using that property?