12.3 Example Function: Split a Line into Words

In this section we illustrate the use of some of the string functions by defining a function to split a line of text into individual words. We have already made use of this function in the concordance example program in Section 9.3.3.

---

NOTE -- The split function is in the concordance program in file concord.cpp.

---

There are three arguments to this function. The first two are strings, describing the line of text and the separators to be used to differentiate words, respectively. The third argument is a list of strings, used to return the individual words in the line.

void split (const std::string& text, const std::string& separators,
            std::list<std::string,
                      std::allocator<std::string> >& words) {

    size_t n     = text.length ();
    size_t start = text.find_first_not_of (separators);

    while (start < n) {
        size_t stop = text.find_first_of (separators, start);
        if (stop > n) stop = n;
        words.push_back (text.substr (start, stop-start));
        start = text.find_first_not_of (separators, stop+1);
    }
}

The program begins by finding start, the index of the first character that is not a separator. The loop then looks for stop, the index of the next character that is a separator, or uses the end of the string if no such value is found. These two indexes delimit a word, which is copied out of the text using a substring operation and inserted into the list of words. A search is then made to discover the start of the next word, and the loop continues. When the index value exceeds the limits of the string, execution stops.