Sometimes we have to handle a workload that is undefined, here I’m going to show how thi could be solved with Do Until and __variables.
The scenario is following: We need to collect all users from a system that has about 14000 users, the problem with this is that due to API restrictions we can collect about 150 users/call. And there is a page handling that we need to use, so we need to get page 1 and page 2 and so on with the page size of 150. In addition to this there is a boolean called last_page to check if we are on the last page.
So how to solve this in Logic Apps? For each cannot be used, calling a function that would collect all users would take to long. Fortunally we can use the new variables actions in combination with the Do_Until action. So what we will do is create a variable to keep the page number, increment it after each GET of users and then check if last_page is true and if it is end the Do until.
The Do Until action is a looping action that continues to execute until a condition is met. Fortunaly Microsoft has provided some limits to make sure we don’t end up in endless loops :) So there are Limits to make sure that the loop is not run more than x times default is set to 60 times or longer than a specified time, as default the timeout is set to 1 hour. But ofcourse we will not need this since we will create good expressions, right? :)
Variables actions provides funtionality to store and handle variables, currently only Integer and float but hopefully soon even objects. There are 3 actions assosiated with variables:
So in order to use variables we need to initialize them via the initalize action and if we want to change the value we can either increment it or set it with the appropiate actions And use them later on in expressions or as in our case as a part of the URL in the HTTP action. We get the value via the following simple code:
Name it to page and set type to Integer and value to 1 since it’s the first page we should start with.
Then we add the Do_Until action using the dountil flag, and also changing the limits since 100 might be to low in the future we set to a really high number of 500 and adding 1 hour in the timout to make sure we can process all users.
As the image shows I’ve used increment by 2 and an @or statment in the Condition, this is since I’m running two parallel executions to speed up the process, fetching 2 * 150 users per turn. So anyone of them could be the last page and if anyone of them is we set the dountil variable to 1 to exit the loop.
Now we have created a flow that will collect users until the system tells us we have collected them all.
This is a good pattern and usefull but there are som considerations to take, when using the Do Until loops we don’t get the “pretty” logging that we get in the For Each loops where we can see all the runs, we just se the latest one.
Variables are really handy and easy to use, makes it possible to keep track of indexes and such when doing more complex workflows.