How to improve performance of ngRepeat over a huge dataset (angular.js)?

How to improve performance of ngRepeat over a huge dataset (angular.js)?

I have a huge dataset of several thousand rows with around 10 fields each, about 2MBs of data. I need to display it in the browser. Most straightforward approach (fetch data, put it into $scope, let ng-repeat=”” do its job) works fine, but it freezes the browser for about half of a minute when it starts inserting nodes into DOM. How should I approach this problem?
One option is to append rows to $scope incrementally and wait for ngRepeat to finish inserting one chunk into DOM before moving to the next one. But AFAIK ngRepeat does not report back when it finishes “repeating”, so it’s going to be ugly.
Other option is to split data on the server into pages and fetch them in multiple requests, but that’s even uglier.
I looked through Angular documentation in search of something like ng-repeat=”data in dataset” ng-repeat-steps=”500″, but found nothing. I am fairly new to Angular ways, so it is possible that I am missing the point completely. What are the best practices at this?

Solutions/Answers:

Solution 1:

I agree with @AndreM96 that the best approach is to display only a limited amount of rows, faster and better UX, this could be done with a pagination or with an infinite scroll.

Infinite scroll with Angular is really simple with limitTo filter. You just have to set the initial limit and when the user asks for more data (I am using a button for simplicity) you increment the limit.

<table>
    <tr ng-repeat="d in data | limitTo:totalDisplayed"><td>{{d}}</td></tr>
</table>
<button class="btn" ng-click="loadMore()">Load more</button>

//the controller
$scope.totalDisplayed = 20;

$scope.loadMore = function () {
  $scope.totalDisplayed += 20;  
};

$scope.data = data;

Here is a JsBin.

Related:  Coordinating parallel execution in node.js

This approach could be a problem for phones because usually they lag when scrolling a lot of data, so in this case I think a pagination fits better.

To do it you will need the limitTo filter and also a custom filter to define the starting point of the data being displayed.

Here is a JSBin with a pagination.

Solution 2:

The hottest – and arguably most scalable – approach to overcoming these challenges with large datasets is embodied by the approach of Ionic’s collectionRepeat directive and of other implementations like it. A fancy term for this is ‘occlusion culling’, but you can sum it up as: don’t just limit the count of rendered DOM elements to an arbitrary (but still high) paginated number like 50, 100, 500… instead, limit only to as many elements as the user can see.

If you do something like what’s commonly known as “infinite scrolling”, you’re reducing the initial DOM count somewhat, but it bloats quickly after a couple refreshes, because all those new elements are just tacked on at the bottom. Scrolling comes to a crawl, because scrolling is all about element count. There’s nothing infinite about it.

Related:  How to create a file in memory for user to download, but not through server?

Whereas, the collectionRepeat approach is to use only as many elements as will fit in viewport, and then recycle them. As one element rotates out of view, it’s detached from the render tree, refilled with data for a new item in the list, then reattached to the render tree at the other end of the list. This is the fastest way known to man to get new information in and out of the DOM, making use of a limited set of existing elements, rather than the traditional cycle of create/destroy… create/destroy. Using this approach, you can truly implement an infinite scroll.

Note that you don’t have to use Ionic to use/hack/adapt collectionRepeat, or any other tool like it. That’s why they call it open-source. 🙂 (That said, the Ionic team is doing some pretty ingenious things, worthy of your attention.)


There’s at least one excellent example of doing something very similar in React. Only instead of recycling the elements with updated content, you’re simply choosing not to render anything in the tree that’s not in view. It’s blazing fast on 5000 items, although their very simple POC implementation allows a bit of flicker…

Related:  Javascript code for showing yesterday's date and todays date

Also… to echo some of the other posts, using track by is seriously helpful, even with smaller datasets. Consider it mandatory.

Solution 3:

I recommend to see this:

Optimizing AngularJS: 1200ms to 35ms

they made a new directive by optimizing ng-repeat at 4 parts:

Optimization#1: Cache DOM elements

Optimization#2: Aggregate watchers

Optimization#3: Defer element creation

Optimization#4: Bypass watchers for hidden elements

the project is here on github:

Usage:

1- include these files in your single-page app:

  • core.js
  • scalyr.js
  • slyEvaluate.js
  • slyRepeat.js

2- add module dependency:

var app = angular.module("app", ['sly']);

3- replace ng-repeat

<tr sly-repeat="m in rows"> .....<tr>

ENjoY!

Solution 4:

Beside all the above hints like track by and smaller loops, this one also helped me a lot

<span ng-bind="::stock.name"></span>

this piece of code would print the name once it has been loaded, and stop watching it after that. Similarly, for ng-repeats, it could be used as

<div ng-repeat="stock in ::ctrl.stocks">{{::stock.name}}</div>

however it only works for AngularJS version 1.3 and higher.
From
http://www.befundoo.com/blog/optimizing-ng-repeat-in-angularjs/

Solution 5:

You can use “track by” to increase the performance:

<div ng-repeat="a in arr track by a.trackingKey">

Faster than:

<div ng-repeat="a in arr">

ref:https://www.airpair.com/angularjs/posts/angularjs-performance-large-applications

Solution 6:

If all your rows have equal height, you should definitely take a look at the virtualizing ng-repeat: http://kamilkp.github.io/angular-vs-repeat/

This demo looks very promising (and it supports inertial scrolling)