Implementing of strtok() function in C++

The strtok() function is used in tokenizing a string based on a delimiter. It is present in the header file “string.h” and returns a pointer to the next token if present, if the next token is not present it returns NULL. To get all the tokens the idea is to call this function in a loop.

Header File:

#include <string.h>

Syntax: 

char *strtok(char *s1, const char *s2);

In this article, we will discuss the implementation of this function in which two things must be taken into consideration:

• Maintain the state of the string to make sure how many tokens we have already extracted.
• Secondly, maintain the list of extracted tokens in an array to return it.

Steps:

• Create a function strtok() which accepts string and delimiter as an argument and return char pointer.
• Create a static variable input to maintain the state of the string.
• Check if extracting the tokens for the first time then initialize the input with it.
• If the input is NULL and all the tokens are extracted then return NULL.
• In this step, start extracting tokens and store them in the array result[].
• Now, iterate a loop until NULL occurs or the delimiter then return the result by including ‘\0’.
• When the end of the string is reached and if it requires then manually add a ‘\0‘ and include this corner case in the end.

Below is the implementation of the same:

C++

// C++ program to demonstrate the function
// strtok() to tokenized the string
#include <bits/stdc++.h>
using namespace std;
char* mystrtok(char* s, char d)
{
    // Stores the state of string
    static char* input = NULL;

    // Initialize the input string
    if (s != NULL)
        input = s;

    // Case for final token
    if (input == NULL)
        return NULL;

    // Stores the extracted string
    char* result = new char[strlen(input) + 1];
    int i = 0;

    // Start extracting string and
    // store it in array
    for (; input[i] != '\0'; i++) {

        // If delimiter is not reached
        // then add the current character
        // to result[i]
        if (input[i] != d)
            result[i] = input[i];

        // Else store the string formed
        else {
            result[i] = '\0';
            input = input + i + 1;
            return result;
        }
    }

    // Case when loop ends
    result[i] = '\0';
    input = NULL;

    // Return the resultant pointer
    // to the string
    return result;
}

// Driver Code
int main()
{
    // Given string str
    char str[90] = "It, is my, day";

    // Tokenized the first string
    char* ptr = mystrtok(str, ' ');

    // Print current tokenized string
    cout << ptr << endl;

    // While ptr is not NULL
    while (ptr != NULL) {
        // Tokenize the string
        ptr = mystrtok(NULL, ' ');

        // Print the string
        cout << ptr << endl;
    }
    return 0;
}

Output:

It,
is
my,
day