Web Service Pagination

[urayoan]

Hello everybody
  I am curious, there is a way or ideas to paginate web service results and send chunks of data per requests?

  I am having trouble with some big results after a request to a web service, and want to know ideas or how to make that work.

Cheers!

[Bruce]

Hi Ura,

there's no "standard" way of dealing with this. Tell me more about the service you are sending to.
Are you sending a lot, or receiving a lot?
Is it your server or someone else's server?

Cheers
Bruce

[urayoan]

Hi Bruce:
 Thank you for the fast reply!

  Is a server on my part and i am sending the data.

The way i am dealing with, is making a view where the date and set filters marks the next position in the request. It works, but i was curious if there's a more simple/effective way to do that.

The service who receives the request has this parameters
SessionToken
AccountNumber
NextDate


The next date is my mark for the next request (the requesters needs to send me the date of the last received record). then with SetFilter and ApplyFilter from the view y get the position for the next set of records.

At the moment is working fine, but i would be interesting to now if there's another way to make the same function or a better approach.

Cheers!

[urayoan]

Oh and i forgot to mention that the date of the most recent record is unknown to the user. In that case the date is empty and the filter is set with the AccountNumber

[Bruce]

Hi Ura,

This approach you have is reasonably common.
Basically if you want the user to be able to fetch a subset of the results, then you give them some sort of parameters to do that.
In your case you're asking them for a "From Date" and then filtering the view based on that date. That's a good solution.

The only issue with this solution is that it is a very "coarse" measurement - so the user will likely get duplicate records during the day. For example, today is 28 June. So during the day I might poll your service 5 or 6 times specifying 28 June. Each time I get all the records for today, obviously many of them I have already.

This is not really a big deal though. The data set is still small.

Of course if a record changes for some reason in the past, then I don't get that change. I'm asking for records from 28 June, but say some other field, on an old record, for 20 June changed, then I don't get that. That may or may not be a problem.

In my own app I use the "server time stamp" value. This contains a timestamp of when the record was last changed (on the server). So using this I can check my own file, see the highest servertimestamp is x, so then ask for everything after x. that way I get "everything that has changed". It's the same basic idea as what you are already doing, but I'm using a date/time stamp (so it's not as coarse as a whole day, and I'm getting it based on the _changed date_ rather than say the invoice date.

Cheers
Bruce

[urayoan]

Thank you Bruce.

The problem with the duplicate records and sometimes records not sent was one of the first problems i was facing making the paging in the web service.

If by example in a specific date, there was 15 records and the page size was specific to 10, when the next date is requested, the client can not get the other 5 records of the previous date. To fix that, if i had more records than the page size, i will deliver the additional records specific to that date (usually a few more records, nothing too large). Of course, this approach is more specific to our DB design how the records are handled.

Thank you again for give me direction and new ideas about this!

Cheers!