MapReduce Unwinding … Reduce

Once this shuffling completed, it is where REDUCE come into action.  Its task is to process the input given by SHUFFLE into the output so that user can understand what is the result of the file processed by hadoop. After shuffling completed, it is clear that one word will be processed by only one DN and not Read more about MapReduce Unwinding … Reduce[…]

MapReduce Unwinding … Sort & Shuffle

This is in continuation of MapReduce Processing …… This output will be input for next process which is SORT. Sort takes this [L<K,V>] and sorts all the words in order of alpha bates (a to z) on each DN. Sorted arrangement on DNs : DN -1: NODE – 1 [ L (K, V)] PKT-1(K) V Read more about MapReduce Unwinding … Sort & Shuffle[…]

MapReduce Unwinding. . . . . Map

In last discussion on MapReduce, we discussed the algorithm which is used by Hadoop for data processing using MapReduce. Now its time to understand this in detail with help of an example. Lets consider our scenario : We have 7 Node cluster where 1 Node is Name Node (NN) and rest of 6 node is Read more about MapReduce Unwinding. . . . . Map[…]

MapReduce Unwinding. . . . . .Algorithm

With discussion, in my last blog, about “How Hadoop manages Fault Tolerance” within its cluster while processing data, it is now time to discuss the algorithm which MapReduce used to process these data. It is Name Node (NN) where a user submits his request to process data and submits his data files.  As soon as NN receives data Read more about MapReduce Unwinding. . . . . .Algorithm[…]

MapReduce Unwinding. . . . . . Fault Tolerance

Before we see the intermediate data produced by the mapper, it would be quite interesting to see the fault tolerant aspects of Hadoop with respect to MapReduce processing. Once Name node (NN) received data files which has to be processed, it splits data files to assign it to Data Node (DN).  This assignment would be Read more about MapReduce Unwinding. . . . . . Fault Tolerance[…]

Tum Laut Jao Priye