Need More Info
Last Updated: 17 Mar 2022 07:38 by ADMIN
Stuart
Created on: 01 Mar 2022 18:24
Category: UI for ASP.NET MVC
Type: Bug Report
0
Kendo Grid server filtering json impact (Not just MVC but any version of the grid)
Server filtering kendo grids and json deserialization errors in asp.net.
When we are managing large volumes of data with several thousand datasource objects (grid rows) all of the rows need to be transported with each event.
This leads to errors with the json being too large to serialize.

We wish to restrict the data footprint and only return data for the page requested.
so if the grid has 100 pages and page size is 10 we only want to return the 10 rows required for the page, not 1000 rows.
This is not possible with the Kendo grid.
To work around this issue we have devised a strategy to return the the number of datasource objects Kendo grid expects up to the requested page count, but they are empty objects with no data, and then only have the last 10 rows populated with data.
It would be better for the grid control just to expect the number of results expected to the selected page and not have to add the dummy rows, so long as the total number of rows is provided so that the grid knows how many pages there are.

Our work around looks something like this in asp.net MVC

var users = cm.getUsers();
var result = new DataSourceResult();
var results = new List<user>();
int index = 0;
if (pageNum == 1)
    results = users.Take(pageSize).ToList();
else
{
     index = pageSize * (pageNum - 1);
     results = users.Skip(index).Take(pageSize).ToList();
}
List<user> output = null;

[This is the bit that should be handled by the kendo grid, without the need for dummy rows to further reduce json size]
if (index == 0)
{
     output = new List<user>(results);
}
else
{
     output = new user[index].ToList();
     output.AddRange(results);
}
result = output.ToDataSourceResult(request);
result.Total = users.Count();
return Json(result, JsonRequestBehavior.AllowGet);
5 comments
ADMIN
Ivan Danchev
Posted on: 17 Mar 2022 07:38

Stuart,

I do not see any issue with the way ToDataSourceResult works. See the attached sample project. The data initially contains 100 records. When it is converted to DataSourceResult, the "result" variable holds 20 records. The information about the page size that comes with the  DataSourceRequest object is taken into account and the correct set of records is returned back to the client.

Could you please modify the example accordingly so that it demonstrates the issue and attach it back for further review?

Regards,
Ivan Danchev
Progress Telerik

Love the Telerik and Kendo UI products and believe more people should try them? Invite a fellow developer to become a Progress customer and each of you can get a $50 Amazon gift voucher.

Stuart
Posted on: 10 Mar 2022 09:01

I see, so the toDataSourceResult trims the results. But we already do that in our code before getting to the toDataSourceResult, and we do that because we need to run additional processes on the raw 20 rows of data before submitting it to the toDataSourceResult. And we don't want to run those additional processes on 10000 rows of data.

Then it is the totoDataSourceResult that is not handling the event that the data may already be of the correct number of rows and does not need trimming.

So using ajax instead of MVC we just need to create the json with total number and the 20 rows required.

 

 

ADMIN
Ivan Danchev
Posted on: 09 Mar 2022 17:52

Hi Stuart,

As I mentioned previously, the server does return only 20 records. Yes the service's Read() method gets all the data, but then the result is converted to DataSourceResult before returning it to the client. This is where only 20 records are picked from the data and only 20 are returned to the client. How many items to get and for which page is information that is available in the read action through the DataSourceRequest object. This way when this information is applied to all the data (by converting it to DataSourceResult), only the data that is needed for that particular page is extracted from it and only 20 records are returned. This can be observed if you look closely into the response of the read request. See the attached screenshot of the response from the demo I linked.

Marked with a red rectangle are the 20 records that are actually returned.

total: 77 pointed by the red arrow is just a number that tells the dataSource how many records are there in the data on the server. But, as can be seen in the screenshot, only the data of 20 of these 77 records are returned. The total number of the records is needed so that the Grid can display a pager with the proper number of pages. In this case 4 pages (20, 20, 20, 17). When you click on page 2, a new request will be sent to the read action and a new batch of 20 items will be returned for page 2, if you click on page 3 another request will be sent and so on.

Regards,
Ivan Danchev
Progress Telerik

Love the Telerik and Kendo UI products and believe more people should try them? Invite a fellow developer to become a Progress customer and each of you can get a $50 Amazon gift voucher.

Attached Files:
Stuart
Posted on: 08 Mar 2022 17:37

Hi Ivan,

in the example you highlighted the server does not return only 20 rows. The server returns all 77 rows.

 

 public ActionResult EditingPopup_Read([DataSourceRequest] DataSourceRequest request)
        {
            return Json(productService.Read().ToDataSourceResult(request));
        }

It just reading from productService. The server is passing all 77 items. Every page, every sort = 77 rows of data.

It is wrong to have to send all 77 records with each request when server filtering. Unnecessary JSON. 

Only the rows needed for the page num should be returned. If I am on page 2, or page 3 or page 4 I want to only send 20 rows. But on if on Page 2 the grid breaks if there are not at least 40 rows. If on page 3 the grid breaks if there are not at least 60 rows.  e.sender.dataSource._pristineData.length == 0 is true. 

What I have devised as a work around is to pad with empty rows, so if on page 2 we have 20 empty rows and 20 full rows, or on page 3 we have 40 empty rows and 20 full rows.

What we do is find the rows in server data for our page :  users.Skip(index).Take(pageSize).ToList();

And then we pad it with empty rows output = new user[index].ToList();
     output.AddRange(results);

 

We shouldn't need to do this. 77 rows of data is quite small, but when dealing with large data of 1000s of rows we often exceed the JSON limit.

 

ADMIN
Ivan Danchev
Posted on: 08 Mar 2022 15:55

Hello Stuart,

Could you elaborate more on the scenario? What causes all of the records data to be sent to the server?

Here's an example of that uses remote binding and editing: https://demos.telerik.com/aspnet-mvc/grid/editing-popup

The page size is set to 20 in its DataSource configuration. If you look at the requests, initially the Grid sends a request to the EditingPopup_Read action and it returns only 20 records (first page). If you go to another page another set of 20 records is returned by the server. If you edit a record, the data of that particular record is sent to the server. At no point the Grid requests or passes to the server the data of all its records.

How does the configuration of your Grid differs from that in the linked demo, so that you get a behavior where all the data is transferred? You can see the configuration of the Grid in the demo's View Source tab. Usually such behavior indicates that ServerOperation is disabled in the DataSource of the Grid, which would cause all the data to be requested initially, instead of specific sets of the data.

Regards,
Ivan Danchev
Progress Telerik

Virtual Classroom, the free self-paced technical training that gets you up to speed with Telerik and Kendo UI products quickly just got a fresh new look + new and improved content including a brand new Blazor course! Check it out at https://learn.telerik.com/.