Reservoir sampling is a family of randomized algorithms for randomly choosing k samples from a list of n items, where n is either a very large or unknown number. Typically n is large enough that the list doesn’t fit into main memory. For example, a list of search queries in Google and Facebook.
So we are given a big array (or stream) of numbers (to simplify), and we need to write an efficient function to randomly select k numbers where 1 <= k <= n. Let the input array be stream.
A simple solution is to create an array reservoir of maximum size k. One by one randomly select an item from stream[0..n-1]. If the selected item is not previously selected, then put it in reservoir. To check if an item is previously selected or not, we need to search the item in reservoir. The time complexity of this algorithm will be O(k^2). This can be costly if k is big. Also, this is not efficient if the input is in the form of a stream.
It can be solved in O(n) time. The solution also suits well for input in the form of stream. The idea is similar to this post. Following are the steps.
1) Create an array reservoir[0..k-1] and copy first k items of stream to it.
2) Now one by one consider all items from (k+1)th item to nth item.
…a) Generate a random number from 0 to i where i is index of current item in stream. Let the generated random number is j.
…b) If j is in range 0 to k-1, replace reservoir[j] with arr[i]
Following is implementation of the above algorithm.
Following are k randomly selected items 6 2 11 8 12 Note: Output will differ every time as it selects and prints random elements
Time Complexity: O(n)
How does this work?
To prove that this solution works perfectly, we must prove that the probability that any item stream[i] where 0 <= i < n will be in final reservoir is k/n. Let us divide the proof in two cases as first k items are treated differently.
Case 1: For last n-k stream items, i.e., for stream[i] where k <= i < n
For every such stream item stream[i], we pick a random index from 0 to i and if the picked index is one of the first k indexes, we replace the element at picked index with stream[i]
To simplify the proof, let us first consider the last item. The probability that the last item is in final reservoir = The probability that one of the first k indexes is picked for last item = k/n (the probability of picking one of the k items from a list of size n)
Let us now consider the second last item. The probability that the second last item is in final reservoir = [Probability that one of the first k indexes is picked in iteration for stream[n-2]] X [Probability that the index picked in iteration for stream[n-1] is not same as index picked for stream[n-2] ] = [k/(n-1)]*[(n-1)/n] = k/n.
Similarly, we can consider other items for all stream items from stream[n-1] to stream[k] and generalize the proof.
Case 2: For first k stream items, i.e., for stream[i] where 0 <= i < k
The first k items are initially copied to reservoir and may be removed later in iterations for stream[k] to stream[n].
The probability that an item from stream[0..k-1] is in final array = Probability that the item is not picked when items stream[k], stream[k+1], …. stream[n-1] are considered = [k/(k+1)] x [(k+1)/(k+2)] x [(k+2)/(k+3)] x … x [(n-1)/n] = k/n
Please write comments if you find anything incorrect, or you want to share more information about the topic discussed above.
Attention reader! Don’t stop learning now. Get hold of all the important DSA concepts with the DSA Self Paced Course at a student-friendly price and become industry ready.
- Count clockwise array rotations required to maximize count of array elements present at indices same as their value
- Check if GCD of all Composite Numbers in an array divisible by K is a Fibonacci Number or not
- Construct smallest N-digit number possible by placing digits at positions specified by given array
- Maximize minimum of array generated by maximums of same indexed elements of two rows of a given Matrix
- Count subsequences for every array element in which they are the maximum
- Predict the winner of a card game of removing K cards in each turn such that Bitwise AND of K and size of pile is 0
- Maximize sum of MEX values of each node in an N-ary Tree
- Smallest pair of indices with product of subarray co-prime with product of the subarray on the left or right
- Maximize count of rows consisting of equal elements by flipping columns of a Matrix
- Partial derivative of a polynomial using Doubly Linked List
- Polynomial Division using Linked List
- Partition array into minimum number of equal length subsets consisting of a single distinct value
- Largest divisor of a number not divisible by another given number
- Minimum cost required to move all elements to the same position