1 <chapter xmlns="http://docbook.org/ns/docbook" version="5.0"
2 xml:id="std.strings" xreflabel="Strings">
3 <?dbhtml filename="strings.html"?>
7 <indexterm><primary>Strings</primary></indexterm>
19 <!-- Sect1 01 : Character Traits -->
21 <!-- Sect1 02 : String Classes -->
22 <section xml:id="std.strings.string" xreflabel="string"><info><title>String Classes</title></info>
25 <section xml:id="strings.string.simple" xreflabel="Simple Transformations"><info><title>Simple Transformations</title></info>
28 Here are Standard, simple, and portable ways to perform common
29 transformations on a <code>string</code> instance, such as
30 "convert to all upper case." The word transformations
31 is especially apt, because the standard template function
32 <code>transform<></code> is used.
35 This code will go through some iterations. Here's a simple
39 #include <string>
40 #include <algorithm>
41 #include <cctype> // old <ctype.h>
45 char operator() (char c) const { return std::tolower(c); }
50 char operator() (char c) const { return std::toupper(c); }
55 std::string s ("Some Kind Of Initial Input Goes Here");
57 // Change everything into upper case
58 std::transform (s.begin(), s.end(), s.begin(), ToUpper());
60 // Change everything into lower case
61 std::transform (s.begin(), s.end(), s.begin(), ToLower());
63 // Change everything back into upper case, but store the
64 // result in a different string
65 std::string capital_s;
66 capital_s.resize(s.size());
67 std::transform (s.begin(), s.end(), capital_s.begin(), ToUpper());
71 <emphasis>Note</emphasis> that these calls all
72 involve the global C locale through the use of the C functions
73 <code>toupper/tolower</code>. This is absolutely guaranteed to work --
74 but <emphasis>only</emphasis> if the string contains <emphasis>only</emphasis> characters
75 from the basic source character set, and there are <emphasis>only</emphasis>
76 96 of those. Which means that not even all English text can be
77 represented (certain British spellings, proper names, and so forth).
78 So, if all your input forevermore consists of only those 96
79 characters (hahahahahaha), then you're done.
81 <para><emphasis>Note</emphasis> that the
82 <code>ToUpper</code> and <code>ToLower</code> function objects
83 are needed because <code>toupper</code> and <code>tolower</code>
84 are overloaded names (declared in <code><cctype></code> and
85 <code><locale></code>) so the template-arguments for
86 <code>transform<></code> cannot be deduced, as explained in
87 <link xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://gcc.gnu.org/ml/libstdc++/2002-11/msg00180.html">this
89 <!-- section 14.8.2.4 clause 16 in ISO 14882:1998 -->
90 At minimum, you can write short wrappers like
95 return std::tolower(c);
97 <para>(Thanks to James Kanze for assistance and suggestions on all of this.)
99 <para>Another common operation is trimming off excess whitespace. Much
100 like transformations, this task is trivial with the use of string's
101 <code>find</code> family. These examples are broken into multiple
102 statements for readability:
105 std::string str (" \t blah blah blah \n ");
107 // trim leading whitespace
108 string::size_type notwhite = str.find_first_not_of(" \t\n");
109 str.erase(0,notwhite);
111 // trim trailing whitespace
112 notwhite = str.find_last_not_of(" \t\n");
113 str.erase(notwhite+1); </programlisting>
114 <para>Obviously, the calls to <code>find</code> could be inserted directly
115 into the calls to <code>erase</code>, in case your compiler does not
116 optimize named temporaries out of existence.
120 <section xml:id="strings.string.case" xreflabel="Case Sensitivity"><info><title>Case Sensitivity</title></info>
125 <para>The well-known-and-if-it-isn't-well-known-it-ought-to-be
126 <link xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://www.gotw.ca/gotw/">Guru of the Week</link>
127 discussions held on Usenet covered this topic in January of 1998.
128 Briefly, the challenge was, <quote>write a 'ci_string' class which
129 is identical to the standard 'string' class, but is
130 case-insensitive in the same way as the (common but nonstandard)
131 C function stricmp()</quote>.
134 ci_string s( "AbCdE" );
137 assert( s == "abcde" );
138 assert( s == "ABCDE" );
140 // still case-preserving, of course
141 assert( strcmp( s.c_str(), "AbCdE" ) == 0 );
142 assert( strcmp( s.c_str(), "abcde" ) != 0 ); </programlisting>
144 <para>The solution is surprisingly easy. The original answer was
145 posted on Usenet, and a revised version appears in Herb Sutter's
146 book <emphasis>Exceptional C++</emphasis> and on his website as <link xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://www.gotw.ca/gotw/029.htm">GotW 29</link>.
148 <para>See? Told you it was easy!</para>
150 <emphasis>Added June 2000:</emphasis> The May 2000 issue of C++
151 Report contains a fascinating <link xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://lafstern.org/matt/col2_new.pdf"> article</link> by
152 Matt Austern (yes, <emphasis>the</emphasis> Matt Austern) on why
153 case-insensitive comparisons are not as easy as they seem, and
154 why creating a class is the <emphasis>wrong</emphasis> way to go
155 about it in production code. (The GotW answer mentions one of
156 the principle difficulties; his article mentions more.)
158 <para>Basically, this is "easy" only if you ignore some things,
159 things which may be too important to your program to ignore. (I chose
160 to ignore them when originally writing this entry, and am surprised
161 that nobody ever called me on it...) The GotW question and answer
162 remain useful instructional tools, however.
164 <para><emphasis>Added September 2000:</emphasis> James Kanze provided a link to a
165 <link xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://www.unicode.org/reports/tr21/tr21-5.html">Unicode
166 Technical Report discussing case handling</link>, which provides some
167 very good information.
171 <section xml:id="strings.string.character_types" xreflabel="Arbitrary Characters"><info><title>Arbitrary Character Types</title></info>
176 <para>The <code>std::basic_string</code> is tantalizingly general, in that
177 it is parameterized on the type of the characters which it holds.
178 In theory, you could whip up a Unicode character class and instantiate
179 <code>std::basic_string<my_unicode_char></code>, or assuming
180 that integers are wider than characters on your platform, maybe just
181 declare variables of type <code>std::basic_string<int></code>.
183 <para>That's the theory. Remember however that basic_string has additional
184 type parameters, which take default arguments based on the character
185 type (called <code>CharT</code> here):
188 template <typename CharT,
189 typename Traits = char_traits<CharT>,
190 typename Alloc = allocator<CharT> >
191 class basic_string { .... };</programlisting>
192 <para>Now, <code>allocator<CharT></code> will probably Do The Right
193 Thing by default, unless you need to implement your own allocator
196 <para>But <code>char_traits</code> takes more work. The char_traits
197 template is <emphasis>declared</emphasis> but not <emphasis>defined</emphasis>.
198 That means there is only
201 template <typename CharT>
204 static void foo (type1 x, type2 y);
207 <para>and functions such as char_traits<CharT>::foo() are not
208 actually defined anywhere for the general case. The C++ standard
209 permits this, because writing such a definition to fit all possible
210 CharT's cannot be done.
212 <para>The C++ standard also requires that char_traits be specialized for
213 instantiations of <code>char</code> and <code>wchar_t</code>, and it
214 is these template specializations that permit entities like
215 <code>basic_string<char,char_traits<char>></code> to work.
217 <para>If you want to use character types other than char and wchar_t,
218 such as <code>unsigned char</code> and <code>int</code>, you will
219 need suitable specializations for them. For a time, in earlier
220 versions of GCC, there was a mostly-correct implementation that
221 let programmers be lazy but it broke under many situations, so it
222 was removed. GCC 3.4 introduced a new implementation that mostly
223 works and can be specialized even for <code>int</code> and other
226 <para>If you want to use your own special character class, then you have
227 <link xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://gcc.gnu.org/ml/libstdc++/2002-08/msg00163.html">a lot
228 of work to do</link>, especially if you with to use i18n features
229 (facets require traits information but don't have a traits argument).
231 <para>Another example of how to specialize char_traits was given <link xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://gcc.gnu.org/ml/libstdc++/2002-08/msg00260.html">on the
232 mailing list</link> and at a later date was put into the file <code>
233 include/ext/pod_char_traits.h</code>. We agree
234 that the way it's used with basic_string (scroll down to main())
235 doesn't look nice, but that's because <link xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://gcc.gnu.org/ml/libstdc++/2002-08/msg00236.html">the
236 nice-looking first attempt</link> turned out to <link xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://gcc.gnu.org/ml/libstdc++/2002-08/msg00242.html">not
237 be conforming C++</link>, due to the rule that CharT must be a POD.
238 (See how tricky this is?)
243 <section xml:id="strings.string.token" xreflabel="Tokenizing"><info><title>Tokenizing</title></info>
247 <para>The Standard C (and C++) function <code>strtok()</code> leaves a lot to
248 be desired in terms of user-friendliness. It's unintuitive, it
249 destroys the character string on which it operates, and it requires
250 you to handle all the memory problems. But it does let the client
251 code decide what to use to break the string into pieces; it allows
252 you to choose the "whitespace," so to speak.
254 <para>A C++ implementation lets us keep the good things and fix those
255 annoyances. The implementation here is more intuitive (you only
256 call it once, not in a loop with varying argument), it does not
257 affect the original string at all, and all the memory allocation
260 <para>It's called stringtok, and it's a template function. Sources are
261 as below, in a less-portable form than it could be, to keep this
262 example simple (for example, see the comments on what kind of
263 string it will accept).
267 #include <string>
268 template <typename Container>
270 stringtok(Container &container, string const &in,
271 const char * const delimiters = " \t\n")
273 const string::size_type len = in.length();
274 string::size_type i = 0;
278 // Eat leading whitespace
279 i = in.find_first_not_of(delimiters, i);
280 if (i == string::npos)
281 return; // Nothing left but white space
283 // Find the end of the token
284 string::size_type j = in.find_first_of(delimiters, i);
287 if (j == string::npos)
289 container.push_back(in.substr(i));
293 container.push_back(in.substr(i, j-i));
295 // Set up for next loop
303 The author uses a more general (but less readable) form of it for
304 parsing command strings and the like. If you compiled and ran this
310 std::list<string> ls;
311 stringtok (ls, " this \t is\t\n a test ");
312 for (std::list<string>const_iterator i = ls.begin();
315 std::cerr << ':' << (*i) << ":\n";
317 <para>You would see this as output:
323 :test: </programlisting>
324 <para>with all the whitespace removed. The original <code>s</code> is still
325 available for use, <code>ls</code> will clean up after itself, and
326 <code>ls.size()</code> will return how many tokens there were.
328 <para>As always, there is a price paid here, in that stringtok is not
329 as fast as strtok. The other benefits usually outweigh that, however.
332 <para><emphasis>Added February 2001:</emphasis> Mark Wilden pointed out that the
333 standard <code>std::getline()</code> function can be used with standard
334 <code>istringstreams</code> to perform
335 tokenizing as well. Build an istringstream from the input text,
336 and then use std::getline with varying delimiters (the three-argument
337 signature) to extract tokens into a string.
342 <section xml:id="strings.string.shrink" xreflabel="Shrink to Fit"><info><title>Shrink to Fit</title></info>
346 <para>From GCC 3.4 calling <code>s.reserve(res)</code> on a
347 <code>string s</code> with <code>res < s.capacity()</code> will
348 reduce the string's capacity to <code>std::max(s.size(), res)</code>.
350 <para>This behaviour is suggested, but not required by the standard. Prior
351 to GCC 3.4 the following alternative can be used instead
354 std::string(str.data(), str.size()).swap(str);
356 <para>This is similar to the idiom for reducing
357 a <code>vector</code>'s memory usage
358 (see <link linkend="faq.size_equals_capacity">this FAQ
359 entry</link>) but the regular copy constructor cannot be used
360 because libstdc++'s <code>string</code> is Copy-On-Write.
362 <para>In <link linkend="status.iso.200x">C++0x</link> mode you can call
363 <code>s.shrink_to_fit()</code> to achieve the same effect as
364 <code>s.reserve(s.size())</code>.
370 <section xml:id="strings.string.Cstring" xreflabel="CString (MFC)"><info><title>CString (MFC)</title></info>
375 <para>A common lament seen in various newsgroups deals with the Standard
376 string class as opposed to the Microsoft Foundation Class called
377 CString. Often programmers realize that a standard portable
378 answer is better than a proprietary nonportable one, but in porting
379 their application from a Win32 platform, they discover that they
380 are relying on special functions offered by the CString class.
382 <para>Things are not as bad as they seem. In
383 <link xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://gcc.gnu.org/ml/gcc/1999-04n/msg00236.html">this
384 message</link>, Joe Buck points out a few very important things:
387 <listitem><para>The Standard <code>string</code> supports all the operations
388 that CString does, with three exceptions.
390 <listitem><para>Two of those exceptions (whitespace trimming and case
391 conversion) are trivial to implement. In fact, we do so
394 <listitem><para>The third is <code>CString::Format</code>, which allows formatting
395 in the style of <code>sprintf</code>. This deserves some mention:
399 The old libg++ library had a function called form(), which did much
400 the same thing. But for a Standard solution, you should use the
401 stringstream classes. These are the bridge between the iostream
402 hierarchy and the string class, and they operate with regular
403 streams seamlessly because they inherit from the iostream
404 hierarchy. An quick example:
407 #include <iostream>
408 #include <string>
409 #include <sstream>
411 string f (string& incoming) // incoming is "foo N"
413 istringstream incoming_stream(incoming);
417 incoming_stream >> the_word // extract "foo"
418 >> the_number; // extract N
420 ostringstream output_stream;
421 output_stream << "The word was " << the_word
422 << " and 3*N was " << (3*the_number);
424 return output_stream.str();
426 <para>A serious problem with CString is a design bug in its memory
427 allocation. Specifically, quoting from that same message:
430 CString suffers from a common programming error that results in
431 poor performance. Consider the following code:
433 CString n_copies_of (const CString& foo, unsigned n)
436 for (unsigned i = 0; i < n; i++)
441 This function is O(n^2), not O(n). The reason is that each +=
442 causes a reallocation and copy of the existing string. Microsoft
443 applications are full of this kind of thing (quadratic performance
444 on tasks that can be done in linear time) -- on the other hand,
445 we should be thankful, as it's created such a big market for high-end
448 If you replace CString with string in the above function, the
451 <para>Joe Buck also pointed out some other things to keep in mind when
452 comparing CString and the Standard string class:
455 <listitem><para>CString permits access to its internal representation; coders
456 who exploited that may have problems moving to <code>string</code>.
458 <listitem><para>Microsoft ships the source to CString (in the files
459 MFC\SRC\Str{core,ex}.cpp), so you could fix the allocation
460 bug and rebuild your MFC libraries.
461 <emphasis><emphasis>Note:</emphasis> It looks like the CString shipped
462 with VC++6.0 has fixed this, although it may in fact have been
463 one of the VC++ SPs that did it.</emphasis>
465 <listitem><para><code>string</code> operations like this have O(n) complexity
466 <emphasis>if the implementors do it correctly</emphasis>. The libstdc++
467 implementors did it correctly. Other vendors might not.
469 <listitem><para>While chapters of the SGI STL are used in libstdc++, their
470 string class is not. The SGI <code>string</code> is essentially
471 <code>vector<char></code> and does not do any reference
472 counting like libstdc++'s does. (It is O(n), though.)
473 So if you're thinking about SGI's string or rope classes,
474 you're now looking at four possibilities: CString, the
475 libstdc++ string, the SGI string, and the SGI rope, and this
476 is all before any allocator or traits customizations! (More
477 choices than you can shake a stick at -- want fries with that?)
484 <!-- Sect1 03 : Interacting with C -->