Yarn resource request normalizaion

I ran in to this issue last week, and it doesn't seem to be documented. This is written with the Fair scheduler in mind but it might be true for the capacity scheduler as well since the setting it self is Yarn global.

When a resource request is send to Yarn, to make management easier the request is normalised before processed. The way that requests are normalised in the case of fair scheduler is by passing the request through the following algorithm. 

min(((max(request, minimum_assign) + (normaliser-1))/normaliser)*normaliser, maximum_assing)

Since minimum and maximum are pretty well known and mentioned in several sites, they are probably already set, however the only place I could find anything about the normaliser was in the source itself.

The default value for the normaliser is set to 1024mb meaning that even if a request is made for 1mb, the request will be normalised to 1024. The same algorithm is also used for vcores, however since the normaliser is set to 1 it is not really an issue.

The value to change the normaliser is configurable using the yarn.scheduler.increment-allocation-mb setting, so it might be advisable to change it to 256 for smaller clusters.